Paragraph length variance

Test: Bad Writing Habits

Avg. Score

92.8%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Mistral Small Creative	100.0%	$0.0007	9.1s	100%
2	Ministral 3 14B	100.0%	$0.0007	11.7s	100%
3	Mistral Small 4	100.0%	$0.0014	18.2s	100%
4	Ministral 8B	99.9%	$0.0004	10.4s	98%
5	Ministral 3 8B	100.0%	$0.0008	19.6s	100%
6	ByteDance Seed 1.6 Flash	100.0%	$0.0013	27.3s	100%
7	Mistral Small 4 (Reasoning)	100.0%	$0.0022	30.2s	100%
8	Mistral Large 3	100.0%	$0.0033	30.3s	100%
9	Ministral 3B	99.5%	$0.0001	8.1s	94%
10	GPT-5.4 Nano (Reasoning)	100.0%	$0.0061	24.5s	99%
11	GPT-5.4 Nano	100.0%	$0.0057	26.3s	100%
12	GPT-5.4 Mini (Reasoning, Low)	100.0%	$0.015	16.8s	100%
13	Writer: Palmyra X5	100.0%	$0.011	22.0s	100%
14	GPT-5.4 Mini	100.0%	$0.015	16.8s	100%
15	Mistral Medium 3.1	100.0%	$0.0048	36.5s	100%
16	GPT-5.4 Nano (Reasoning, Low)	99.8%	$0.0055	20.6s	96%
17	Mistral Large 2	99.9%	$0.013	29.4s	99%
18	Gemini 3 Flash (Preview)	99.3%	$0.0078	19.6s	94%
19	Gemma 3 27B	99.9%	$0.0006	52.6s	98%
20	Qwen3 235B A22B Instruct 2507	100.0%	$0.0011	59.2s	100%
21	GPT-5.4 Mini (Reasoning)	100.0%	$0.022	28.1s	100%
22	Stealth: Hunter Alpha	99.7%	$0.0000	55.0s	97%
23	Z.AI GLM 5 Turbo	99.7%	$0.0081	33.2s	94%
24	MiniMax M2.7	100.0%	$0.0040	1.1m	100%
25	Gemini 3 Flash (Preview, Reasoning)	99.3%	$0.012	30.1s	94%
26	Ministral 3 3B	98.6%	$0.0005	11.1s	85%
27	Gemma 3 12B	98.9%	$0.0004	41.3s	92%
28	MiniMax M2.5	100.0%	$0.0034	1.3m	100%
29	Mistral Large	99.6%	$0.014	30.9s	93%
30	Qwen 3.5 Flash	99.4%	$0.0025	47.5s	93%
31	Claude Sonnet 4.6	100.0%	$0.031	39.3s	100%
32	Qwen 3.5 Plus (2026-02-15)	98.0%	$0.0060	31.5s	89%
33	GPT-4.1 Mini	97.6%	$0.0027	19.0s	85%
34	Claude Haiku 4.5	99.0%	$0.011	21.6s	87%
35	Qwen 3 32B	99.0%	$0.0015	54.6s	91%
36	DeepSeek V3 (2025-03-24)	98.9%	$0.0014	39.4s	87%
37	Stealth: Healer Alpha	97.2%	$0.0000	23.7s	84%
38	Aion 2.0	99.7%	$0.0064	1.3m	95%
39	Z.AI GLM 4.7	99.8%	$0.010	1.4m	97%
40	Gemma 3 4B	97.2%	$0.0002	20.0s	79%
41	Claude Sonnet 4.5	99.4%	$0.035	38.1s	93%
42	Qwen 3.5 35B	99.3%	$0.018	1.0m	93%
43	Z.AI GLM 4.7 Flash	99.0%	$0.0017	1.2m	88%
44	DeepSeek V3.2	99.7%	$0.0014	1.9m	96%
45	GPT-4.1	98.3%	$0.018	44.7s	87%
46	Z.AI GLM 5	99.0%	$0.0084	1.2m	87%
47	DeepSeek V3 (2024-12-26)	97.9%	$0.0021	54.6s	82%
48	Claude 3.5 Haiku	94.5%	$0.0035	10.8s	77%
49	Claude 3.7 Sonnet	99.0%	$0.042	46.7s	93%
50	Grok 4.1 Fast	96.7%	$0.0018	37.8s	79%
51	Gemini 3.1 Flash Lite (Preview)	94.5%	$0.0030	8.4s	76%
52	DeepSeek-V2 Chat	97.5%	$0.0021	53.3s	79%
53	Grok 4.20 (Beta)	96.2%	$0.018	15.8s	77%
54	Qwen 3.5 122B	98.8%	$0.025	1.1m	88%
55	Claude Opus 4.5	100.0%	$0.070	53.4s	100%
56	GPT-5.4	100.0%	$0.049	1.4m	100%
57	Gemini 3 Pro (Preview)	99.7%	$0.055	54.4s	94%
58	Claude Sonnet 4.6 (Reasoning)	100.0%	$0.060	1.2m	100%
59	DeepSeek V3.1	98.6%	$0.0020	1.8m	88%
60	GPT-5.4 (Reasoning, Low)	100.0%	$0.055	1.4m	100%
61	GPT-5.2	100.0%	$0.056	1.5m	100%
62	Qwen 3.5 9B	97.5%	$0.0011	1.4m	79%
63	Z.AI GLM 4.5	93.8%	$0.0051	42.1s	75%
64	GPT-5.1	100.0%	$0.054	1.8m	100%
65	Z.AI GLM 4.6	95.9%	$0.0065	51.5s	74%
66	Claude Opus 4.6	100.0%	$0.078	1.2m	100%
67	Qwen 3.5 27B	99.1%	$0.020	1.6m	84%
68	GPT-4o Mini (temp=0)	94.5%	$0.0012	34.8s	69%
69	GPT-5 Mini	95.2%	$0.0100	57.4s	77%
70	Gemini 2.5 Pro	96.4%	$0.036	36.2s	78%
71	Grok 4.20 (Beta, Reasoning)	96.1%	$0.039	34.0s	78%
72	GPT-4o Mini (temp=1)	91.1%	$0.0012	34.8s	71%
73	Claude Sonnet 4	95.3%	$0.032	43.7s	77%
74	Claude Opus 4.6 (Reasoning)	100.0%	$0.088	1.4m	100%
75	Qwen 2.5 72B	91.0%	$0.0010	36.7s	67%
76	MoonshotAI: Kimi K2.5	99.7%	$0.019	3.2m	96%
77	Qwen 3.5 397B A17B	98.6%	$0.014	3.0m	90%
78	Arcee AI: Trinity Large (Preview)	88.4%	$0.0000	43.6s	67%
79	Gemini 2.5 Flash (Reasoning)	89.1%	$0.011	21.5s	63%
80	GPT-4o, May 13th (temp=1)	89.7%	$0.033	14.4s	69%
81	GPT-5	100.0%	$0.065	2.8m	100%
82	LFM2 24B	91.6%	$0.0002	28.4s	51%
83	Llama 3.1 8B	91.7%	$0.0003	1.3m	60%
84	GPT-5.4 (Reasoning)	100.0%	$0.089	2.6m	100%
85	Gemini 2.5 Flash	86.7%	$0.0052	10.6s	54%
86	o4 Mini	88.6%	$0.015	25.7s	55%
87	GPT-4.1 Nano	83.7%	$0.0007	13.3s	54%
88	GPT-4o, May 13th (temp=0)	89.6%	$0.035	14.1s	56%
89	Gemini 3.1 Pro (Preview)	99.0%	$0.107	1.8m	90%
90	Rocinante 12B	83.5%	$0.0014	38.4s	52%
91	Gemini 2.5 Flash Lite	82.0%	$0.0009	9.5s	46%
92	Grok 4 Fast	82.5%	$0.0017	24.1s	48%
93	Claude 3.5 Sonnet	86.7%	$0.048	35.5s	62%
94	GPT-4o, Aug. 6th (temp=0)	82.3%	$0.023	22.7s	51%
95	WizardLM 2 8x22b	88.5%	$0.0026	1.8m	53%
96	Llama 3.1 70B	79.4%	$0.0015	29.4s	45%
97	o4 Mini High	85.9%	$0.025	47.2s	47%
98	GPT-4o, Aug. 6th (temp=1)	79.5%	$0.018	24.4s	50%
99	Mistral NeMO	77.6%	$0.0005	10.1s	41%
100	Gemini 2.5 Flash Lite (Reasoning)	79.7%	$0.0028	30.8s	43%
101	Cohere Command R+ (Aug. 2024)	82.1%	$0.020	52.5s	50%
102	Hermes 3 405B	78.0%	$0.0032	53.2s	48%
103	Hermes 3 70B	79.8%	$0.0010	1.2m	47%
104	Inception Mercury 2	74.5%	$0.0032	7.0s	41%
105	Grok 4	86.7%	$0.048	1.7m	59%
106	Nemotron 3 Super	79.9%	$0.0000	1.4m	47%
107	Llama 3.1 Nemotron 70B	76.1%	$0.0038	31.7s	41%
108	ByteDance Seed 1.6	86.1%	$0.013	2.5m	56%
109	Claude 3 Haiku	72.6%	$0.0025	14.9s	38%
110	Claude Opus 4	98.5%	$0.209	1.4m	89%
111	Stealth: Aurora Alpha	69.2%	$0.0000	9.8s	38%
112	ByteDance Seed 2.0 Lite	82.0%	$0.012	2.2m	48%
113	GPT-5 Nano	65.5%	$0.0042	1.4m	43%
114	Arcee AI: Trinity Mini	64.3%	$0.0003	9.2s	20%
115	Nemotron 3 Nano	61.6%	$0.0010	1.1m	27%
116	ByteDance Seed 2.0 Mini	81.0%	$0.0045	4.9m	48%
117	Inception Mercury	54.9%	$0.011	17.6s	15%
118	Mistral Small 3.2 24B	61.8%	$0.0069	5.7m	20%
92.82%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	99	99.8%
GPT-4o Mini (temp=0)	100	100	100	100	98	99.6%
Gemma 3 27B	100	100	100	100	97	99.5%
Nemotron 3 Super	100	100	100	100	97	99.3%
Claude Sonnet 4	100	100	100	100	96	99.2%
DeepSeek-V2 Chat	100	100	100	100	94	98.8%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	92	98.3%
Gemini 2.5 Flash Lite	100	100	100	100	91	98.2%
Claude Opus 4	100	100	100	100	91	98.2%
o4 Mini	100	100	100	100	91	98.1%
GPT-4o, May 13th (temp=1)	100	100	100	98	91	97.9%
Gemini 2.5 Flash	100	100	100	97	92	97.8%
Grok 4.20 (Beta)	100	100	100	100	88	97.6%
Qwen 2.5 72B	100	100	100	93	91	96.9%
Claude 3.7 Sonnet	100	100	100	100	84	96.8%
o4 Mini High	100	100	100	100	83	96.6%
Stealth: Healer Alpha	100	100	100	90	85	95.0%
Gemma 3 12B	100	100	100	99	75	94.8%
Qwen 3.5 35B	100	100	100	100	71	94.2%
Claude 3.5 Haiku	100	100	98	88	85	94.0%
Arcee AI: Trinity Large (Preview)	100	100	98	89	83	93.8%
GPT-4o, May 13th (temp=0)	100	100	100	90	79	93.8%
Z.AI GLM 4.5	100	100	100	86	82	93.6%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	68	93.6%
Claude 3.5 Sonnet	100	100	91	87	83	92.2%
Z.AI GLM 4.6	100	100	100	85	71	91.2%
GPT-4o Mini (temp=1)	100	100	90	87	78	91.0%
Grok 4	100	100	100	79	70	89.7%
Gemini 2.5 Flash Lite (Reasoning)	100	100	93	78	71	88.3%
GPT-4o, Aug. 6th (temp=0)	100	100	86	81	62	85.8%
Hermes 3 405B	100	100	83	83	59	84.9%
Rocinante 12B	100	100	100	63	60	84.5%
Hermes 3 70B	100	99	95	70	55	83.9%
Ministral 3 3B	100	100	100	75	36	82.3%
Mistral NeMO	100	96	93	76	47	82.3%
ByteDance Seed 1.6	100	100	75	73	61	81.7%
GPT-4o, Aug. 6th (temp=1)	100	81	79	75	73	81.7%
Llama 3.1 70B	100	100	83	69	56	81.6%
Mistral Small 3.2 24B	100	100	90	67	50	81.3%
GPT-4.1 Nano	98	97	83	63	62	80.4%
Claude 3 Haiku	100	100	82	67	52	80.1%
GPT-5 Nano	97	85	76	69	65	78.2%
Llama 3.1 Nemotron 70B	88	86	83	74	55	77.0%
Cohere Command R+ (Aug. 2024)	100	100	91	78	13	76.5%
Llama 3.1 8B	100	100	100	53	28	76.3%
Inception Mercury	100	86	82	69	42	75.8%
ByteDance Seed 2.0 Mini	100	80	79	53	45	71.5%
ByteDance Seed 2.0 Lite	100	100	76	53	27	71.2%
Nemotron 3 Nano	100	88	61	59	45	70.7%
Inception Mercury 2	97	94	81	28	24	64.6%
Stealth: Aurora Alpha	97	69	48	42	41	59.2%
Arcee AI: Trinity Mini	80	47	32	15	0	34.9%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	99	99.8%
Ministral 3 3B	100	100	100	100	96	99.2%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	94	98.7%
Arcee AI: Trinity Large (Preview)	100	100	100	100	88	97.6%
Claude 3.5 Haiku	100	100	100	100	87	97.5%
Llama 3.1 8B	100	100	100	95	91	97.1%
Qwen 3.5 122B	100	100	100	100	77	95.3%
Gemini 2.5 Pro	100	100	100	100	75	95.0%
ByteDance Seed 1.6	100	100	100	100	74	94.7%
Grok 4.20 (Beta)	100	100	100	100	55	91.0%
Arcee AI: Trinity Mini	100	100	100	100	53	90.6%
GPT-4o Mini (temp=0)	100	100	99	95	56	90.0%
ByteDance Seed 2.0 Lite	100	100	100	100	49	89.8%
DeepSeek V3 (2024-12-26)	100	100	100	100	46	89.2%
Z.AI GLM 4.5	100	98	93	87	65	88.5%
Gemini 2.5 Flash	100	100	100	92	49	88.2%
Llama 3.1 Nemotron 70B	100	100	100	82	49	86.2%
GPT-4o Mini (temp=1)	100	100	100	62	59	84.0%
Z.AI GLM 4.6	100	100	100	100	20	83.9%
Rocinante 12B	100	100	100	100	0	80.0%
Inception Mercury 2	100	100	84	71	44	79.6%
Gemini 2.5 Flash Lite (Reasoning)	100	85	84	69	46	76.8%
Gemma 3 4B	100	100	100	49	31	75.9%
Hermes 3 405B	99	92	64	61	54	74.0%
Gemini 2.5 Flash (Reasoning)	89	80	76	61	60	73.1%
o4 Mini High	100	100	74	47	44	72.9%
GPT-4o, May 13th (temp=1)	100	83	65	59	50	71.4%
Nemotron 3 Super	100	74	65	61	56	71.2%
Stealth: Aurora Alpha	87	72	71	65	52	69.4%
GPT-4.1 Nano	100	74	74	63	34	69.1%
o4 Mini	87	75	74	62	45	68.5%
Grok 4.1 Fast	100	75	66	52	46	68.0%
Cohere Command R+ (Aug. 2024)	100	81	74	60	24	67.8%
Gemini 2.5 Flash Lite	96	92	88	57	4	67.3%
Mistral Small 3.2 24B	100	100	68	50	0	63.8%
Mistral NeMO	91	87	86	49	6	63.6%
Qwen 2.5 72B	100	82	62	40	34	63.5%
GPT-4o, Aug. 6th (temp=1)	81	66	57	56	41	60.0%
Hermes 3 70B	100	86	69	31	0	57.0%
GPT-4o, May 13th (temp=0)	76	72	72	62	0	56.5%
Llama 3.1 70B	100	94	50	17	14	54.9%
GPT-5 Nano	70	59	47	47	42	53.0%
ByteDance Seed 2.0 Mini	100	84	51	17	11	52.5%
Grok 4	77	64	39	29	18	45.4%
Grok 4 Fast	63	48	43	43	26	44.5%
Claude 3 Haiku	100	81	18	12	10	44.3%
GPT-4o, Aug. 6th (temp=0)	61	53	44	34	21	42.7%
Nemotron 3 Nano	55	41	26	18	18	31.5%
Inception Mercury	46	21	13	0	0	16.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	99	99.8%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	99	99.7%
GPT-5.4 Nano	100	100	100	100	98	99.5%
Claude 3.5 Haiku	100	100	100	100	97	99.5%
Llama 3.1 Nemotron 70B	100	100	100	100	96	99.2%
Qwen 3 32B	100	100	100	100	95	98.9%
Grok 4.20 (Beta)	100	100	100	99	94	98.8%
GPT-4.1 Nano	100	100	100	98	92	98.2%
GPT-4o Mini (temp=0)	100	100	100	97	93	98.1%
DeepSeek V3 (2024-12-26)	100	100	100	100	89	97.8%
GPT-4o Mini (temp=1)	100	100	100	96	91	97.5%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	84	96.8%
ByteDance Seed 2.0 Mini	100	100	97	94	92	96.4%
GPT-4o, May 13th (temp=1)	100	100	100	95	87	96.3%
Grok 4.1 Fast	100	100	100	100	73	94.7%
Mistral NeMO	100	100	100	96	71	93.3%
Cohere Command R+ (Aug. 2024)	100	100	100	100	66	93.2%
Gemma 3 4B	100	100	100	100	65	93.1%
Hermes 3 405B	100	100	100	100	65	93.1%
Llama 3.1 70B	100	100	100	86	79	93.0%
Gemini 2.5 Flash Lite	100	100	100	84	79	92.6%
Arcee AI: Trinity Large (Preview)	100	100	100	89	70	91.8%
Claude 3 Haiku	100	100	95	90	73	91.5%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	57	91.3%
GPT-4o, Aug. 6th (temp=1)	100	96	95	83	80	90.9%
Rocinante 12B	100	100	100	78	73	90.2%
Hermes 3 70B	100	100	88	87	74	89.7%
Inception Mercury 2	100	100	100	75	63	87.5%
ByteDance Seed 1.6	100	98	94	94	50	87.3%
Arcee AI: Trinity Mini	100	100	99	93	39	86.1%
Grok 4	100	100	99	75	57	86.1%
Nemotron 3 Super	100	100	94	75	58	85.3%
Grok 4 Fast	100	100	83	76	67	85.2%
Stealth: Aurora Alpha	100	90	84	62	52	77.7%
Nemotron 3 Nano	100	84	71	60	48	72.6%
GPT-5 Nano	85	81	71	71	54	72.5%
Mistral Small 3.2 24B	85	84	60	53	7	57.8%
Inception Mercury	100	67	25	17	0	41.8%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	96	99.3%
Gemma 3 12B	100	100	100	100	96	99.2%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	96	99.1%
Nemotron 3 Super	100	100	100	100	95	98.9%
Stealth: Healer Alpha	100	100	100	100	95	98.9%
GPT-4o Mini (temp=1)	100	100	100	100	93	98.5%
GPT-4o, May 13th (temp=1)	100	100	100	100	91	98.3%
GPT-4.1 Nano	100	100	100	96	93	97.9%
Claude 3.5 Sonnet	100	100	100	100	89	97.7%
Grok 4 Fast	100	100	100	95	93	97.5%
GPT-4o, Aug. 6th (temp=0)	100	100	100	97	89	97.2%
Z.AI GLM 4.5	100	100	100	100	85	97.1%
Claude 3.5 Haiku	100	100	100	100	85	97.1%
Grok 4	100	100	100	100	84	96.8%
GPT-4o Mini (temp=0)	100	100	100	95	88	96.6%
Qwen 2.5 72B	100	100	100	100	81	96.2%
Gemini 2.5 Flash Lite	100	100	100	100	78	95.6%
Qwen 3.5 9B	100	100	100	100	74	94.9%
GPT-4o, May 13th (temp=0)	100	100	100	90	79	93.8%
Hermes 3 70B	100	100	100	89	75	92.8%
Llama 3.1 8B	100	100	100	87	76	92.5%
Claude 3 Haiku	100	100	100	100	61	92.3%
Hermes 3 405B	100	100	100	86	74	92.0%
Llama 3.1 70B	100	100	96	88	75	91.8%
Arcee AI: Trinity Large (Preview)	100	100	100	80	72	90.4%
ByteDance Seed 1.6	100	100	100	96	54	90.0%
Gemini 2.5 Flash	100	100	99	73	70	88.5%
Cohere Command R+ (Aug. 2024)	100	100	100	97	40	87.5%
Llama 3.1 Nemotron 70B	100	100	97	83	54	86.8%
ByteDance Seed 2.0 Mini	100	100	90	72	67	85.9%
Qwen 3.5 27B	100	100	100	100	26	85.2%
Mistral Small 3.2 24B	100	95	87	85	41	81.5%
Arcee AI: Trinity Mini	100	100	100	100	6	81.2%
GPT-4o, Aug. 6th (temp=1)	99	85	77	74	62	79.3%
GPT-5 Nano	87	83	81	73	71	79.1%
ByteDance Seed 2.0 Lite	100	100	98	48	48	78.8%
Gemini 2.5 Flash Lite (Reasoning)	100	100	88	70	25	76.6%
Inception Mercury 2	100	94	71	64	49	75.8%
Nemotron 3 Nano	100	98	86	81	12	75.3%
Inception Mercury	100	100	79	47	42	73.5%
Stealth: Aurora Alpha	100	70	64	46	26	61.2%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	98	99.7%
Ministral 3B	100	100	100	100	98	99.6%
Claude 3.5 Sonnet	100	100	100	100	97	99.4%
Gemma 3 12B	100	100	100	100	96	99.2%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	96	99.1%
Stealth: Healer Alpha	100	100	100	100	95	99.0%
Qwen 3.5 9B	100	100	100	100	90	98.1%
Gemma 3 4B	100	100	100	100	88	97.6%
Z.AI GLM 4.5	100	100	100	100	88	97.6%
GPT-4o, Aug. 6th (temp=0)	100	100	99	99	88	97.1%
Cohere Command R+ (Aug. 2024)	100	100	100	97	86	96.6%
o4 Mini High	100	100	100	100	81	96.2%
GPT-4o Mini (temp=1)	100	100	98	92	86	95.2%
ByteDance Seed 2.0 Mini	100	100	100	100	76	95.1%
Qwen 3 32B	100	100	100	100	72	94.4%
Llama 3.1 8B	100	100	100	100	70	94.0%
GPT-4o, May 13th (temp=1)	100	100	100	99	70	93.8%
Nemotron 3 Super	100	100	99	85	84	93.7%
Mistral NeMO	100	100	100	93	76	93.7%
Grok 4	100	100	100	97	68	93.1%
Llama 3.1 70B	100	100	100	83	57	88.1%
GPT-4o, Aug. 6th (temp=1)	100	100	94	85	60	88.0%
Gemini 2.5 Flash Lite	100	100	86	82	71	87.7%
Hermes 3 405B	100	100	86	85	60	86.3%
ByteDance Seed 1.6	100	100	94	76	59	85.8%
Arcee AI: Trinity Large (Preview)	100	100	84	71	65	84.0%
Hermes 3 70B	100	100	80	73	64	83.3%
Claude 3 Haiku	100	99	94	80	39	82.3%
GPT-4o Mini (temp=0)	94	88	85	71	70	81.7%
Grok 4 Fast	100	100	81	71	57	81.6%
Inception Mercury	100	100	100	72	24	79.2%
GPT-4.1 Nano	93	83	78	73	61	77.6%
Arcee AI: Trinity Mini	100	100	95	57	36	77.5%
Rocinante 12B	100	100	100	63	17	76.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	72	53	51	75.2%
Mistral Small 3.2 24B	100	100	85	79	0	72.9%
GPT-5 Nano	95	84	69	67	49	72.9%
Stealth: Aurora Alpha	83	77	73	61	60	70.7%
ByteDance Seed 2.0 Lite	100	86	70	59	34	69.8%
Inception Mercury 2	79	73	71	58	56	67.4%
Llama 3.1 Nemotron 70B	100	73	66	60	22	64.0%
Nemotron 3 Nano	100	72	61	61	26	63.9%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	98	99.7%
Gemini 2.5 Pro	100	100	100	100	97	99.5%
Claude Sonnet 4	100	100	100	100	87	97.5%
Z.AI GLM 4.5	100	100	100	100	86	97.3%
o4 Mini	100	100	100	100	86	97.3%
Qwen 2.5 72B	100	100	100	99	87	97.3%
Ministral 3B	100	100	100	100	85	97.1%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	85	97.0%
Grok 4	100	100	100	99	86	97.0%
Grok 4 Fast	100	100	100	100	85	96.9%
GPT-4o Mini (temp=1)	100	100	100	100	81	96.2%
GPT-4o, May 13th (temp=0)	100	100	96	91	90	95.4%
Cohere Command R+ (Aug. 2024)	100	100	100	99	73	94.4%
Arcee AI: Trinity Large (Preview)	100	100	100	96	74	94.1%
DeepSeek-V2 Chat	100	100	100	100	65	93.1%
Claude 3.5 Haiku	100	100	100	100	66	93.1%
GPT-4.1 Mini	100	100	100	100	64	92.8%
Claude 3.5 Sonnet	100	100	98	93	71	92.4%
GPT-4.1 Nano	100	100	100	88	69	91.5%
Stealth: Aurora Alpha	100	100	95	83	77	91.2%
Hermes 3 405B	100	100	100	92	63	91.0%
Llama 3.1 Nemotron 70B	100	100	100	79	71	90.0%
Llama 3.1 8B	100	100	100	100	45	88.9%
GPT-4o, May 13th (temp=1)	100	99	90	77	70	87.2%
Hermes 3 70B	100	100	100	77	58	87.0%
Arcee AI: Trinity Mini	100	100	100	100	31	86.2%
Qwen 3.5 9B	100	100	100	84	41	85.2%
Z.AI GLM 4.6	100	100	100	92	34	85.1%
LFM2 24B	100	100	100	100	24	84.8%
GPT-4o, Aug. 6th (temp=1)	100	100	84	71	62	83.4%
Claude 3 Haiku	100	100	82	67	66	82.9%
Nemotron 3 Nano	100	86	86	79	59	81.8%
Nemotron 3 Super	100	94	88	68	56	81.2%
ByteDance Seed 1.6	100	100	75	68	58	80.3%
Gemini 2.5 Flash	100	99	87	64	41	77.9%
Rocinante 12B	100	82	78	66	61	77.5%
GPT-4o, Aug. 6th (temp=0)	100	89	66	64	54	74.7%
Inception Mercury 2	100	80	75	65	48	73.6%
ByteDance Seed 2.0 Mini	100	84	73	70	33	71.9%
Llama 3.1 70B	100	100	85	61	13	71.9%
Inception Mercury	100	85	78	53	27	68.6%
GPT-5 Nano	71	70	67	65	60	66.5%
Mistral NeMO	87	61	33	27	25	46.3%
Gemini 2.5 Flash Lite	81	68	56	11	0	43.2%
Mistral Small 3.2 24B	73	67	26	0	0	33.2%
Gemini 2.5 Flash Lite (Reasoning)	57	46	30	27	5	32.9%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	99	99.6%
DeepSeek V3.2	100	100	100	100	95	99.1%
DeepSeek V3.1	100	100	100	100	95	99.0%
Ministral 3B	100	100	100	100	94	98.8%
Qwen 3.5 Flash	100	100	100	100	92	98.3%
GPT-4o, May 13th (temp=0)	100	100	100	100	90	98.1%
Stealth: Hunter Alpha	100	100	100	100	90	98.0%
Qwen 3 32B	100	100	100	100	89	97.7%
DeepSeek V3 (2025-03-24)	100	100	100	100	88	97.5%
Stealth: Healer Alpha	100	100	100	97	90	97.4%
GPT-4o Mini (temp=0)	100	100	100	97	90	97.4%
GPT-4.1	100	100	99	97	90	97.0%
Z.AI GLM 4.7 Flash	100	100	100	95	90	97.0%
Z.AI GLM 4.7	100	100	100	96	86	96.4%
Gemma 3 12B	100	100	100	100	82	96.4%
Qwen 3.5 397B A17B	100	100	100	100	82	96.4%
Gemini 2.5 Flash (Reasoning)	100	100	99	95	87	96.3%
Ministral 3 3B	100	100	100	100	80	95.9%
Grok 4.1 Fast	100	100	100	100	78	95.6%
Gemini 2.5 Flash	100	100	100	100	76	95.2%
Claude 3.7 Sonnet	100	100	100	95	80	95.0%
Claude Sonnet 4.5	100	100	100	97	77	94.7%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	86	84	94.0%
Qwen 3.5 122B	100	100	100	100	69	93.7%
Z.AI GLM 4.5	100	100	98	96	70	92.7%
Gemma 3 4B	100	100	99	97	64	92.2%
Llama 3.1 8B	100	100	100	87	68	90.9%
Qwen 3.5 Plus (2026-02-15)	100	100	96	89	70	90.9%
Gemini 3.1 Pro (Preview)	100	100	100	82	72	90.8%
Z.AI GLM 4.6	100	100	89	85	76	90.0%
Gemini 3 Flash (Preview)	100	91	88	85	83	89.5%
ByteDance Seed 1.6	100	100	93	78	75	89.2%
DeepSeek-V2 Chat	100	100	100	83	62	89.0%
GPT-4.1 Mini	100	100	95	80	62	87.3%
Gemini 2.5 Flash Lite	100	100	92	76	68	87.3%
GPT-4o, May 13th (temp=1)	100	100	94	79	64	87.3%
Arcee AI: Trinity Large (Preview)	98	98	92	87	58	86.6%
Gemini 2.5 Flash Lite (Reasoning)	100	90	79	77	75	84.2%
GPT-4o Mini (temp=1)	100	100	94	80	46	83.9%
o4 Mini	87	84	84	82	80	83.4%
Qwen 2.5 72B	100	100	86	66	57	81.7%
Rocinante 12B	100	100	100	53	50	80.6%
GPT-5 Mini	100	92	81	69	56	79.6%
ByteDance Seed 2.0 Mini	100	100	86	59	49	78.7%
Hermes 3 70B	100	92	81	74	46	78.5%
Claude 3 Haiku	100	86	82	74	49	78.3%
GPT-4o, Aug. 6th (temp=0)	89	84	73	69	67	76.7%
Claude 3.5 Sonnet	100	86	77	69	39	74.3%
Claude Sonnet 4	96	87	69	63	55	74.1%
Grok 4	100	90	73	70	35	73.6%
LFM2 24B	100	100	100	67	0	73.4%
Gemini 3.1 Flash Lite (Preview)	100	78	71	67	50	73.3%
Grok 4 Fast	100	87	86	73	19	73.0%
Nemotron 3 Super	97	76	73	53	44	68.7%
Mistral NeMO	94	84	65	55	44	68.4%
Llama 3.1 70B	98	69	58	58	43	65.3%
Claude 3.5 Haiku	84	73	64	51	51	64.6%
Hermes 3 405B	100	77	60	55	30	64.3%
GPT-5 Nano	74	68	63	60	56	64.2%
Cohere Command R+ (Aug. 2024)	92	91	54	38	34	61.9%
GPT-4.1 Nano	87	76	66	56	22	61.3%
Arcee AI: Trinity Mini	100	93	66	26	22	61.1%
Inception Mercury 2	73	66	65	58	36	59.5%
Nemotron 3 Nano	64	61	60	52	34	54.0%
o4 Mini High	100	85	71	0	0	51.1%
ByteDance Seed 2.0 Lite	89	80	58	15	11	50.5%
GPT-4o, Aug. 6th (temp=1)	61	51	49	46	41	49.6%
Stealth: Aurora Alpha	52	51	48	42	42	46.8%
Llama 3.1 Nemotron 70B	85	47	28	25	16	40.1%
WizardLM 2 8x22b	53	41	39	27	14	34.8%
Inception Mercury	56	47	24	11	0	27.5%
Mistral Small 3.2 24B	43	5	5	0	0	10.5%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	99.9%
Qwen 3.5 35B	100	100	100	100	99	99.8%
Gemini 2.5 Pro	100	100	100	100	95	99.1%
Qwen 3.5 397B A17B	100	100	100	100	90	98.1%
Stealth: Hunter Alpha	100	100	100	99	90	97.6%
DeepSeek V3.1	100	100	100	100	87	97.4%
Claude 3.7 Sonnet	100	100	100	93	91	96.9%
Gemma 3 4B	100	100	100	100	82	96.5%
DeepSeek-V2 Chat	100	100	100	98	74	94.3%
Z.AI GLM 4.5	100	100	94	91	85	93.9%
Gemini 3.1 Flash Lite (Preview)	100	100	100	95	74	93.9%
ByteDance Seed 2.0 Lite	100	100	100	100	69	93.7%
Claude Sonnet 4	100	100	100	92	71	92.6%
Qwen 3.5 Plus (2026-02-15)	100	100	87	87	82	91.2%
Gemini 2.5 Flash	100	98	96	86	74	91.0%
Z.AI GLM 4.6	100	100	100	100	47	89.5%
GPT-4.1	100	100	100	75	70	89.0%
Arcee AI: Trinity Large (Preview)	97	92	91	83	80	88.5%
Grok 4.20 (Beta, Reasoning)	100	100	100	80	62	88.3%
Stealth: Healer Alpha	100	100	100	67	67	86.9%
Grok 4.1 Fast	100	100	96	78	58	86.5%
DeepSeek V3 (2024-12-26)	100	100	98	60	58	83.3%
Gemini 2.5 Flash (Reasoning)	98	95	95	70	44	80.5%
Llama 3.1 8B	100	100	100	100	2	80.3%
LFM2 24B	100	100	100	89	11	80.0%
Qwen 2.5 72B	100	100	83	57	56	79.2%
GPT-5 Mini	100	94	72	61	55	76.5%
Cohere Command R+ (Aug. 2024)	100	85	82	67	38	74.4%
GPT-4o, May 13th (temp=1)	100	88	80	64	27	71.7%
GPT-4o Mini (temp=1)	81	72	70	70	46	68.0%
Rocinante 12B	100	68	65	54	49	67.2%
Hermes 3 70B	100	100	65	57	1	64.6%
Gemini 2.5 Flash Lite	91	57	56	51	46	60.2%
Llama 3.1 Nemotron 70B	94	88	52	34	31	59.8%
Nemotron 3 Super	84	83	65	45	15	58.1%
Gemini 2.5 Flash Lite (Reasoning)	100	79	71	34	0	57.0%
Hermes 3 405B	85	58	56	48	37	56.8%
GPT-4.1 Nano	100	65	64	33	22	56.8%
Mistral Small 3.2 24B	100	89	40	29	24	56.4%
Claude 3.5 Sonnet	89	81	49	30	27	55.1%
GPT-4o, Aug. 6th (temp=1)	89	65	49	46	8	51.5%
Grok 4	100	57	36	35	27	51.1%
ByteDance Seed 2.0 Mini	90	55	47	46	15	50.5%
GPT-4o Mini (temp=0)	100	86	35	25	0	49.3%
GPT-4o, May 13th (temp=0)	100	68	47	18	5	47.6%
Mistral NeMO	100	53	33	31	21	47.4%
GPT-4o, Aug. 6th (temp=0)	78	58	50	21	14	44.3%
o4 Mini	94	90	18	15	0	43.5%
WizardLM 2 8x22b	73	50	38	31	23	43.0%
Stealth: Aurora Alpha	67	56	44	29	18	42.9%
Nemotron 3 Nano	48	45	44	39	31	41.3%
Inception Mercury 2	67	47	38	35	15	40.4%
ByteDance Seed 1.6	100	70	14	9	6	39.9%
o4 Mini High	81	41	34	24	14	38.8%
Llama 3.1 70B	67	50	25	20	17	35.9%
Claude 3 Haiku	67	44	35	20	0	33.1%
GPT-5 Nano	42	29	28	26	20	29.1%
Grok 4 Fast	56	47	31	6	0	27.9%
Inception Mercury	30	29	23	4	0	17.1%
Arcee AI: Trinity Mini	23	5	3	0	0	6.1%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	98	99.7%
Llama 3.1 70B	100	100	100	100	98	99.6%
Llama 3.1 8B	100	100	100	100	97	99.4%
Mistral Small 3.2 24B	100	100	100	100	95	98.9%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	94	98.7%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	92	98.4%
GPT-4o Mini (temp=1)	100	100	100	100	91	98.3%
ByteDance Seed 2.0 Mini	100	100	100	97	92	97.8%
GPT-4o, Aug. 6th (temp=1)	100	100	100	97	91	97.6%
Gemini 2.5 Flash	100	100	100	100	87	97.5%
Claude Sonnet 4	100	100	100	100	86	97.3%
GPT-5 Mini	100	100	100	100	86	97.2%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	86	97.2%
Arcee AI: Trinity Large (Preview)	100	100	100	99	86	97.1%
Qwen 3.5 Plus (2026-02-15)	100	100	100	98	80	95.6%
Grok 4 Fast	100	100	100	100	73	94.5%
ByteDance Seed 1.6	100	100	100	100	68	93.7%
Grok 4	100	100	100	84	77	92.3%
WizardLM 2 8x22b	100	100	100	84	77	92.2%
Llama 3.1 Nemotron 70B	100	100	88	87	81	91.1%
Mistral NeMO	100	100	100	96	55	90.3%
Hermes 3 70B	100	100	83	82	81	89.1%
GPT-4.1 Nano	100	100	89	85	47	84.3%
Stealth: Aurora Alpha	100	94	87	84	52	83.5%
Hermes 3 405B	100	100	91	70	34	78.9%
Nemotron 3 Super	100	92	77	77	42	77.7%
Rocinante 12B	100	100	71	59	51	76.2%
Nemotron 3 Nano	99	87	68	66	55	74.9%
Claude 3 Haiku	100	89	84	64	36	74.8%
ByteDance Seed 2.0 Lite	100	80	74	58	57	73.9%
Inception Mercury 2	89	75	72	69	62	73.4%
Cohere Command R+ (Aug. 2024)	89	78	77	63	59	73.2%
Arcee AI: Trinity Mini	100	73	61	50	48	66.2%
GPT-5 Nano	80	75	67	63	39	64.6%
Inception Mercury	100	100	68	47	3	63.7%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	98	99.5%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	97	99.4%
Gemini 3 Flash (Preview)	100	100	100	100	96	99.3%
Grok 4 Fast	100	100	100	100	95	99.0%
Llama 3.1 70B	100	100	100	100	95	99.0%
GPT-4o Mini (temp=0)	100	100	100	100	94	98.8%
GPT-4o, May 13th (temp=0)	100	100	100	100	94	98.8%
Claude Opus 4	100	100	100	100	89	97.8%
Arcee AI: Trinity Large (Preview)	100	100	100	100	88	97.7%
Claude Haiku 4.5	100	100	100	100	87	97.3%
GPT-4o Mini (temp=1)	100	100	100	95	92	97.3%
DeepSeek V3.1	100	100	100	100	86	97.2%
Mistral NeMO	100	100	100	100	85	97.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	98	80	95.6%
Qwen 3.5 Flash	100	100	100	100	76	95.2%
Qwen 3.5 397B A17B	100	100	100	100	76	95.1%
Inception Mercury 2	100	100	100	100	74	94.8%
Grok 4	100	100	100	89	84	94.6%
Claude 3.5 Sonnet	100	100	100	91	81	94.4%
Claude 3.5 Haiku	100	100	99	99	71	93.9%
Rocinante 12B	100	100	98	89	81	93.6%
GPT-5 Mini	100	100	100	100	68	93.6%
o4 Mini	100	100	100	85	80	93.2%
Claude 3 Haiku	100	100	100	82	69	90.1%
Gemini 2.5 Flash Lite (Reasoning)	100	100	91	86	71	89.8%
GPT-4o, May 13th (temp=1)	100	100	91	88	65	88.8%
ByteDance Seed 2.0 Mini	100	100	100	89	55	88.8%
Llama 3.1 Nemotron 70B	100	100	84	81	77	88.4%
LFM2 24B	100	100	100	96	44	88.0%
Llama 3.1 8B	100	100	100	74	66	88.0%
Nemotron 3 Super	100	100	90	81	69	87.9%
WizardLM 2 8x22b	100	100	100	80	60	87.9%
Stealth: Aurora Alpha	100	100	85	83	68	87.1%
Qwen 2.5 72B	100	100	99	72	64	86.9%
Mistral Small 3.2 24B	100	100	92	73	56	84.3%
Hermes 3 70B	100	100	96	76	47	83.8%
Gemini 3.1 Flash Lite (Preview)	100	100	75	71	68	82.9%
Cohere Command R+ (Aug. 2024)	100	100	75	69	69	82.7%
GPT-4o, Aug. 6th (temp=1)	100	97	77	69	67	82.0%
ByteDance Seed 1.6	100	100	100	60	46	81.2%
o4 Mini High	100	100	100	100	0	80.0%
Inception Mercury	100	100	75	73	51	79.6%
Hermes 3 405B	100	100	69	62	57	77.4%
Arcee AI: Trinity Mini	100	100	99	28	13	68.0%
Nemotron 3 Nano	87	79	69	60	41	67.3%
ByteDance Seed 2.0 Lite	100	98	63	63	3	65.5%
GPT-5 Nano	72	52	50	49	43	53.3%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	99	99.7%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	98	99.6%
Gemini 3 Flash (Preview)	100	100	100	100	98	99.5%
Claude 3.5 Sonnet	100	100	100	100	94	98.7%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	92	98.4%
Qwen 3 32B	100	100	100	100	91	98.1%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	97	94	98.1%
DeepSeek-V2 Chat	100	100	100	100	90	98.1%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	95	93	97.6%
GPT-4o Mini (temp=0)	100	100	100	95	92	97.5%
Gemma 3 12B	100	100	100	100	86	97.1%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	86	97.1%
o4 Mini	100	100	100	100	84	96.7%
GPT-4.1 Mini	100	100	100	93	91	96.7%
Gemini 2.5 Flash (Reasoning)	100	100	95	94	92	96.2%
Claude 3.5 Haiku	100	100	100	93	85	95.5%
Grok 4.1 Fast	100	100	100	100	77	95.4%
Llama 3.1 70B	100	100	100	90	86	95.2%
Qwen 2.5 72B	100	100	100	98	77	94.9%
Z.AI GLM 4.5	100	100	97	90	81	93.5%
ByteDance Seed 2.0 Mini	100	100	98	87	81	93.1%
Gemini 2.5 Flash	100	100	100	95	67	92.3%
GPT-4o, May 13th (temp=1)	100	100	100	80	77	91.4%
Hermes 3 70B	100	100	100	78	78	91.2%
Gemini 2.5 Flash Lite	100	100	88	85	83	91.1%
Arcee AI: Trinity Large (Preview)	100	100	100	100	52	90.5%
Cohere Command R+ (Aug. 2024)	100	100	100	83	67	90.0%
Rocinante 12B	100	100	100	100	47	89.4%
Grok 4	100	100	96	89	61	89.2%
GPT-5 Mini	100	100	96	92	57	89.0%
Grok 4 Fast	100	100	89	85	69	88.5%
LFM2 24B	100	100	100	92	46	87.7%
Mistral NeMO	100	100	89	78	71	87.7%
GPT-4o Mini (temp=1)	100	100	88	75	73	87.2%
Stealth: Aurora Alpha	100	100	100	84	48	86.5%
GPT-4o, Aug. 6th (temp=1)	100	100	98	80	53	86.2%
Mistral Small 3.2 24B	100	99	90	85	53	85.3%
Nemotron 3 Super	100	89	83	81	58	82.3%
Inception Mercury 2	91	90	82	79	67	81.9%
Hermes 3 405B	92	89	84	81	61	81.5%
ByteDance Seed 1.6	100	100	79	51	43	74.4%
ByteDance Seed 2.0 Lite	97	88	74	73	38	74.0%
GPT-4.1 Nano	88	79	75	73	39	70.7%
Claude 3 Haiku	100	80	76	63	33	70.3%
Inception Mercury	100	100	58	44	43	68.9%
Llama 3.1 Nemotron 70B	100	79	65	51	30	65.0%
Arcee AI: Trinity Mini	88	78	63	61	14	60.6%
Nemotron 3 Nano	97	61	58	40	39	59.0%
GPT-5 Nano	55	54	51	50	45	51.0%
WizardLM 2 8x22b	61	59	43	40	33	47.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	95	99.0%
Gemma 3 27B	100	100	100	100	95	99.0%
Qwen 3.5 27B	100	100	100	97	97	98.8%
Gemini 3 Flash (Preview)	100	100	100	98	96	98.7%
Mistral Large 2	100	100	100	100	93	98.6%
Gemini 3.1 Pro (Preview)	100	100	100	98	93	98.3%
Ministral 3 3B	100	100	100	100	91	98.3%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	90	98.0%
Claude Sonnet 4.5	100	100	100	100	87	97.5%
GPT-4o Mini (temp=0)	100	100	100	100	87	97.5%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	84	96.7%
MoonshotAI: Kimi K2.5	100	100	100	100	81	96.2%
DeepSeek V3.1	100	100	100	100	80	96.0%
GPT-4.1	100	100	100	100	80	96.0%
Qwen 3.5 Flash	100	100	100	100	76	95.1%
Inception Mercury 2	100	100	100	97	79	95.0%
Z.AI GLM 5 Turbo	100	100	100	100	70	93.9%
Qwen 3.5 35B	100	98	97	95	81	93.9%
GPT-4.1 Mini	100	100	100	98	71	93.6%
Z.AI GLM 4.6	100	100	100	85	80	93.0%
Claude Opus 4	100	100	100	89	70	91.9%
Qwen 3.5 122B	100	100	100	84	58	88.5%
Gemini 2.5 Pro	100	100	91	79	72	88.4%
Claude 3.5 Haiku	100	95	90	84	68	87.4%
WizardLM 2 8x22b	100	100	100	100	36	87.2%
GPT-5 Mini	100	100	100	69	63	86.5%
Cohere Command R+ (Aug. 2024)	100	97	93	87	51	85.5%
Qwen 3.5 397B A17B	100	87	83	79	76	85.0%
GPT-4o, May 13th (temp=1)	100	100	99	65	56	83.9%
Arcee AI: Trinity Large (Preview)	100	100	88	80	51	83.8%
Grok 4	100	100	96	67	55	83.5%
Qwen 2.5 72B	100	94	93	69	59	82.9%
Z.AI GLM 5	100	100	100	71	44	82.9%
Stealth: Healer Alpha	100	100	90	65	59	82.9%
LFM2 24B	100	100	100	100	13	82.6%
DeepSeek V3 (2025-03-24)	100	100	100	58	54	82.4%
Nemotron 3 Super	92	91	83	69	69	80.8%
Z.AI GLM 4.5	100	98	89	59	47	78.7%
Claude 3.5 Sonnet	89	89	86	68	60	78.4%
GPT-4o, May 13th (temp=0)	100	100	80	65	35	76.0%
Gemini 2.5 Flash (Reasoning)	100	93	64	64	59	76.0%
Arcee AI: Trinity Mini	100	100	87	69	22	75.7%
Gemini 3.1 Flash Lite (Preview)	100	100	58	57	55	74.1%
Rocinante 12B	100	100	83	67	20	74.0%
ByteDance Seed 1.6	100	77	72	69	49	73.3%
Stealth: Aurora Alpha	81	77	68	68	63	71.3%
Grok 4 Fast	100	91	60	58	46	71.0%
Hermes 3 70B	100	76	75	61	42	70.8%
GPT-4o Mini (temp=1)	100	77	73	58	36	68.7%
GPT-4o, Aug. 6th (temp=1)	97	95	78	41	31	68.5%
Claude Sonnet 4	93	78	61	55	49	67.1%
Hermes 3 405B	100	84	61	59	30	66.6%
Claude 3 Haiku	100	66	55	54	52	65.4%
GPT-4.1 Nano	100	100	64	25	25	62.6%
Gemini 2.5 Flash Lite (Reasoning)	94	71	63	43	37	61.7%
o4 Mini High	97	85	58	36	14	58.1%
Mistral NeMO	79	78	56	49	25	57.4%
Gemini 2.5 Flash Lite	86	61	53	48	36	56.8%
ByteDance Seed 2.0 Mini	76	65	55	49	30	55.1%
Llama 3.1 Nemotron 70B	100	95	65	5	0	53.1%
GPT-4o, Aug. 6th (temp=0)	98	64	44	37	6	49.9%
o4 Mini	100	90	38	21	0	49.8%
Mistral Small 3.2 24B	100	71	39	33	5	49.7%
Gemini 2.5 Flash	64	62	60	45	14	49.1%
Nemotron 3 Nano	80	80	52	30	4	49.0%
Inception Mercury	100	66	47	10	0	44.4%
Llama 3.1 70B	85	44	37	31	23	44.0%
ByteDance Seed 2.0 Lite	99	33	32	31	20	42.8%
GPT-5 Nano	52	40	34	31	30	37.5%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	99	99.8%
Z.AI GLM 4.7	100	100	100	100	98	99.5%
Gemma 3 27B	100	100	100	100	95	99.0%
DeepSeek-V2 Chat	100	100	100	100	95	99.0%
Qwen 3 32B	100	100	100	100	94	98.8%
GPT-5 Mini	100	100	100	99	94	98.6%
Gemini 2.5 Pro	100	100	100	100	93	98.5%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	92	98.4%
Gemini 2.5 Flash	100	100	100	100	91	98.2%
GPT-4.1	100	100	100	94	94	97.7%
Stealth: Healer Alpha	100	100	100	100	88	97.7%
Grok 4	100	100	100	96	91	97.5%
Claude Haiku 4.5	100	100	100	100	85	97.0%
Claude Sonnet 4.5	100	100	100	100	83	96.6%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	82	96.5%
ByteDance Seed 1.6	100	100	100	100	82	96.3%
DeepSeek V3.2	100	100	100	99	83	96.3%
Claude Sonnet 4	100	100	100	99	80	95.8%
GPT-4o Mini (temp=0)	100	100	100	90	88	95.7%
Grok 4 Fast	100	100	100	100	78	95.6%
Llama 3.1 8B	100	100	100	100	77	95.4%
Z.AI GLM 4.6	100	100	100	100	77	95.4%
GPT-4o Mini (temp=1)	100	100	100	100	76	95.3%
Qwen 3.5 Plus (2026-02-15)	100	100	93	93	88	94.8%
Gemini 3 Pro (Preview)	100	100	100	100	73	94.6%
GPT-4.1 Nano	100	100	96	94	83	94.5%
Gemini 2.5 Flash (Reasoning)	100	100	100	92	80	94.3%
Claude 3.7 Sonnet	100	100	91	91	88	94.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	70	94.0%
GPT-4.1 Mini	100	100	100	96	71	93.4%
Claude 3.5 Haiku	100	98	95	89	73	90.8%
Gemini 2.5 Flash Lite	100	100	95	88	70	90.6%
GPT-4o, May 13th (temp=0)	100	100	93	90	69	90.2%
o4 Mini High	100	100	95	81	73	89.6%
Qwen 2.5 72B	100	100	92	80	72	88.9%
ByteDance Seed 2.0 Lite	100	100	100	94	49	88.7%
Claude Opus 4	100	100	94	77	63	87.0%
Z.AI GLM 4.5	100	95	92	87	56	86.0%
DeepSeek V3.1	100	96	96	88	49	85.9%
o4 Mini	100	96	82	77	72	85.3%
Rocinante 12B	100	100	100	83	42	85.0%
Z.AI GLM 4.7 Flash	100	100	100	75	50	85.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	96	28	84.8%
Arcee AI: Trinity Large (Preview)	100	93	83	81	57	82.7%
Cohere Command R+ (Aug. 2024)	100	93	83	68	64	81.6%
Llama 3.1 70B	100	99	81	73	43	79.2%
Claude 3 Haiku	97	94	92	58	55	79.0%
Hermes 3 70B	100	100	93	58	18	73.9%
Nemotron 3 Super	100	100	81	62	24	73.4%
Grok 4.20 (Beta)	100	81	74	59	53	73.4%
GPT-4o, Aug. 6th (temp=1)	100	79	74	65	48	73.3%
ByteDance Seed 2.0 Mini	93	75	75	61	59	72.7%
Inception Mercury	92	76	75	72	40	71.1%
Mistral NeMO	100	85	73	60	33	70.2%
GPT-5 Nano	85	74	72	70	47	69.7%
Hermes 3 405B	82	81	75	68	40	69.1%
Llama 3.1 Nemotron 70B	73	70	67	66	51	65.5%
Claude 3.5 Sonnet	76	71	66	56	45	62.8%
Stealth: Aurora Alpha	84	76	59	50	13	56.7%
Nemotron 3 Nano	100	100	76	1	0	55.2%
Inception Mercury 2	76	72	67	35	11	52.4%
Mistral Small 3.2 24B	64	61	60	44	25	50.8%
Arcee AI: Trinity Mini	96	57	45	4	0	40.6%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	97	99.3%
Gemma 3 12B	100	100	100	100	97	99.3%
Claude Opus 4	100	100	100	100	95	98.9%
MoonshotAI: Kimi K2.5	100	100	100	100	93	98.6%
Gemini 2.5 Pro	100	100	100	100	89	97.7%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	81	96.2%
ByteDance Seed 2.0 Lite	100	100	100	100	69	93.8%
GPT-4o Mini (temp=1)	100	100	100	100	65	93.1%
ByteDance Seed 1.6	100	100	100	100	65	93.0%
GPT-5 Mini	100	100	100	100	64	92.8%
DeepSeek V3 (2024-12-26)	100	100	100	100	59	91.8%
Grok 4	100	100	100	100	55	90.9%
GPT-4.1	100	100	100	100	54	90.8%
ByteDance Seed 2.0 Mini	100	100	99	86	54	87.8%
Qwen 3.5 9B	100	100	100	100	39	87.8%
Llama 3.1 Nemotron 70B	95	94	90	75	74	85.7%
GPT-4o, Aug. 6th (temp=1)	100	100	84	74	70	85.6%
Grok 4.20 (Beta, Reasoning)	100	100	100	94	32	85.1%
DeepSeek-V2 Chat	100	100	100	100	18	83.5%
Inception Mercury 2	100	83	83	78	65	81.9%
o4 Mini	100	100	76	69	62	81.5%
Llama 3.1 70B	100	88	88	62	60	79.7%
GPT-4o, May 13th (temp=1)	100	100	71	67	54	78.5%
Z.AI GLM 4.5	100	100	77	71	39	77.4%
Arcee AI: Trinity Large (Preview)	100	79	76	72	59	77.0%
Qwen 2.5 72B	100	94	86	82	23	76.9%
Hermes 3 70B	100	95	92	66	25	75.5%
o4 Mini High	100	100	79	53	39	74.2%
Hermes 3 405B	89	88	75	70	49	74.2%
GPT-4o, Aug. 6th (temp=0)	88	86	76	67	45	72.5%
GPT-4.1 Nano	100	77	74	53	49	70.7%
Stealth: Aurora Alpha	89	78	69	60	45	68.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	77	31	25	66.5%
GPT-5 Nano	75	74	74	59	51	66.5%
GPT-4o, May 13th (temp=0)	97	93	58	49	18	63.0%
Gemini 2.5 Flash Lite	100	74	67	53	18	62.5%
Rocinante 12B	100	74	53	50	18	59.2%
Cohere Command R+ (Aug. 2024)	99	82	66	40	0	57.4%
Claude 3.5 Sonnet	77	76	55	49	26	56.6%
LFM2 24B	100	100	66	0	0	53.2%
Arcee AI: Trinity Mini	100	100	46	8	0	50.7%
Nemotron 3 Super	100	87	25	22	14	49.7%
Nemotron 3 Nano	100	97	36	0	0	46.6%
Grok 4 Fast	100	51	37	17	16	44.2%
Gemini 2.5 Flash (Reasoning)	78	59	41	20	19	43.3%
Mistral NeMO	85	59	44	9	5	40.4%
Claude 3 Haiku	61	54	41	27	2	37.1%
Gemini 2.5 Flash	67	44	37	27	0	35.1%
Mistral Small 3.2 24B	34	26	18	6	0	16.8%
Inception Mercury	49	20	12	0	0	16.3%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	99	99.8%
GPT-5 Mini	100	100	100	100	99	99.7%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	89	97.9%
Rocinante 12B	100	100	100	96	90	97.3%
Claude 3.5 Sonnet	100	100	100	94	92	97.2%
Mistral Small 3.2 24B	100	100	100	100	83	96.7%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	80	96.0%
Grok 4 Fast	100	100	100	91	88	95.7%
Llama 3.1 70B	100	100	100	100	77	95.5%
Gemini 2.5 Flash Lite	100	100	100	100	75	95.0%
Hermes 3 405B	100	100	100	100	62	92.5%
Llama 3.1 Nemotron 70B	100	100	100	87	72	91.9%
Llama 3.1 8B	100	100	100	89	66	91.0%
Grok 4	100	100	95	79	70	88.9%
GPT-5 Nano	100	100	98	73	59	86.0%
Arcee AI: Trinity Large (Preview)	100	100	76	74	72	84.4%
Grok 4.20 (Beta, Reasoning)	100	88	87	76	69	83.9%
Nemotron 3 Super	100	100	78	77	63	83.7%
Cohere Command R+ (Aug. 2024)	100	100	98	75	39	82.4%
Hermes 3 70B	100	100	81	67	62	82.0%
Grok 4.20 (Beta)	100	100	100	67	33	79.9%
Mistral NeMO	100	100	63	58	55	75.3%
Inception Mercury 2	100	100	83	38	38	71.8%
Inception Mercury	100	91	86	38	24	67.9%
Claude 3 Haiku	75	74	65	52	43	61.9%
Arcee AI: Trinity Mini	100	83	63	59	0	61.0%
Stealth: Aurora Alpha	99	63	54	48	32	59.2%
Nemotron 3 Nano	70	65	64	55	11	52.9%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	98	99.6%
GPT-5.4 Nano (Reasoning)	100	100	100	100	97	99.4%
Gemma 3 4B	100	100	100	100	96	99.2%
Mistral NeMO	100	100	100	100	95	99.0%
GPT-4.1 Nano	100	100	100	100	95	99.0%
GPT-4.1 Mini	100	100	100	100	95	98.9%
Gemini 2.5 Flash Lite	100	100	100	100	94	98.9%
Grok 4.20 (Beta)	100	100	100	100	93	98.6%
Cohere Command R+ (Aug. 2024)	100	100	100	100	94	98.6%
GPT-4o Mini (temp=0)	100	100	100	97	95	98.4%
Z.AI GLM 4.5	100	100	100	96	95	98.1%
Gemma 3 12B	100	100	100	100	87	97.5%
Inception Mercury 2	100	100	100	100	85	97.0%
Qwen 2.5 72B	100	100	100	93	91	96.7%
Nemotron 3 Super	100	100	100	94	88	96.3%
Ministral 3B	100	100	100	100	76	95.2%
o4 Mini High	100	100	100	98	75	94.7%
Inception Mercury	100	100	100	96	75	94.3%
Grok 4 Fast	100	100	100	91	75	93.4%
Gemini 3.1 Pro (Preview)	100	100	100	98	68	93.2%
Gemini 2.5 Flash Lite (Reasoning)	100	100	93	88	83	92.8%
Arcee AI: Trinity Large (Preview)	100	100	100	82	77	91.7%
Arcee AI: Trinity Mini	100	100	100	87	69	91.2%
ByteDance Seed 2.0 Lite	100	100	100	81	71	90.4%
GPT-4o, May 13th (temp=1)	100	100	100	86	63	89.8%
ByteDance Seed 2.0 Mini	100	100	100	95	53	89.6%
Claude 3.5 Haiku	100	100	100	91	55	89.2%
ByteDance Seed 1.6	100	100	100	100	44	88.9%
Claude 3 Haiku	100	100	87	81	75	88.6%
Gemini 3.1 Flash Lite (Preview)	100	100	100	80	61	88.3%
Hermes 3 405B	100	100	91	86	59	87.1%
Claude 3.5 Sonnet	100	100	100	72	63	87.0%
Hermes 3 70B	100	100	92	71	54	83.3%
Llama 3.1 Nemotron 70B	100	98	89	79	49	83.2%
GPT-4o, Aug. 6th (temp=1)	100	91	90	86	45	82.3%
Stealth: Aurora Alpha	100	84	82	77	67	82.0%
Nemotron 3 Nano	100	89	71	66	58	76.8%
GPT-5 Nano	90	78	73	66	45	70.3%
Mistral Small 3.2 24B	100	100	52	49	9	62.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	99	99.8%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	98	99.5%
o4 Mini	100	100	100	100	97	99.4%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	97	99.3%
DeepSeek-V2 Chat	100	100	100	100	96	99.3%
Claude Sonnet 4	100	100	100	100	94	98.8%
o4 Mini High	100	100	100	100	94	98.7%
GPT-4.1 Nano	100	100	100	100	92	98.4%
Grok 4.20 (Beta)	100	100	100	100	92	98.4%
Qwen 2.5 72B	100	100	100	100	90	97.9%
Gemma 3 12B	100	100	100	94	94	97.5%
Mistral NeMO	100	100	100	99	87	97.2%
Stealth: Healer Alpha	100	100	100	100	85	97.0%
GPT-4.1 Mini	100	100	100	100	85	97.0%
Gemma 3 4B	100	100	100	100	85	97.0%
Grok 4 Fast	100	100	100	100	83	96.6%
GPT-4o, May 13th (temp=1)	100	100	98	94	90	96.4%
ByteDance Seed 2.0 Mini	100	100	100	100	75	94.9%
ByteDance Seed 2.0 Lite	100	100	99	92	75	93.1%
Cohere Command R+ (Aug. 2024)	100	100	100	100	62	92.3%
Rocinante 12B	100	100	100	100	60	91.9%
Gemini 2.5 Flash Lite	100	100	100	96	61	91.5%
Grok 4	100	96	92	90	80	91.5%
GPT-4o Mini (temp=1)	100	100	93	83	79	91.1%
GPT-4o, Aug. 6th (temp=0)	100	100	91	83	78	90.3%
GPT-4o, Aug. 6th (temp=1)	100	100	94	78	74	89.3%
Gemini 2.5 Flash	100	97	97	82	68	89.0%
Llama 3.1 70B	100	100	100	96	46	88.4%
Gemini 2.5 Flash Lite (Reasoning)	100	100	98	78	64	88.0%
Arcee AI: Trinity Large (Preview)	100	92	87	86	72	87.5%
Gemini 2.5 Flash (Reasoning)	92	91	88	84	83	87.3%
Grok 4.20 (Beta, Reasoning)	100	100	97	71	64	86.5%
Nemotron 3 Super	100	100	91	75	65	86.2%
GPT-5 Nano	100	94	86	71	70	84.1%
Hermes 3 405B	100	96	80	78	59	82.7%
Claude 3 Haiku	96	93	79	78	56	80.3%
Hermes 3 70B	100	88	75	70	64	79.5%
Llama 3.1 8B	100	100	100	57	18	74.9%
Inception Mercury 2	100	89	75	64	32	72.0%
Mistral Small 3.2 24B	100	95	71	57	0	64.6%
Nemotron 3 Nano	100	77	48	21	19	53.0%
Stealth: Aurora Alpha	82	53	48	36	29	49.7%
Llama 3.1 Nemotron 70B	65	57	57	48	21	49.6%
Arcee AI: Trinity Mini	51	48	39	38	22	39.4%
Inception Mercury	100	45	39	6	0	38.1%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	99	99.8%
DeepSeek V3.2	100	100	100	100	96	99.3%
GPT-4o, May 13th (temp=0)	100	100	100	100	96	99.2%
Z.AI GLM 4.6	100	100	100	100	94	98.9%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	94	98.7%
Gemma 3 4B	100	100	100	100	93	98.6%
Claude 3.5 Haiku	100	100	100	100	92	98.4%
o4 Mini	100	100	100	98	92	98.1%
Ministral 8B	100	100	100	100	90	97.9%
GPT-4.1 Mini	100	100	100	94	93	97.5%
Qwen 2.5 72B	100	100	100	95	92	97.3%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	86	97.1%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	85	97.0%
Stealth: Healer Alpha	100	100	100	98	82	96.0%
o4 Mini High	100	100	100	100	77	95.5%
Aion 2.0	100	100	100	100	74	94.8%
Z.AI GLM 4.5	100	100	100	100	74	94.8%
GPT-4o Mini (temp=1)	100	100	100	89	84	94.6%
Grok 4.20 (Beta)	100	100	100	91	79	94.1%
Qwen 3 32B	100	100	100	100	68	93.7%
Mistral Large	100	100	100	100	68	93.6%
Claude Sonnet 4	100	100	100	100	66	93.2%
GPT-4.1 Nano	100	100	100	87	78	93.0%
Llama 3.1 Nemotron 70B	100	100	96	91	76	92.7%
Qwen 3.5 9B	100	100	100	100	48	89.6%
Claude Haiku 4.5	100	100	100	100	39	87.8%
Mistral NeMO	100	100	98	82	59	87.7%
GPT-4o, May 13th (temp=1)	100	100	100	77	60	87.5%
GPT-4o, Aug. 6th (temp=1)	100	92	88	85	65	85.9%
Stealth: Aurora Alpha	100	100	100	81	32	82.7%
Nemotron 3 Nano	94	87	82	82	67	82.4%
Cohere Command R+ (Aug. 2024)	100	100	100	69	38	81.5%
GPT-5 Nano	100	98	75	74	59	81.3%
Llama 3.1 8B	100	100	100	86	19	81.2%
Arcee AI: Trinity Mini	100	100	99	67	38	80.9%
GPT-4o, Aug. 6th (temp=0)	100	100	80	64	58	80.3%
Rocinante 12B	100	100	97	62	40	79.6%
Claude 3 Haiku	94	81	81	72	49	75.6%
ByteDance Seed 2.0 Mini	100	100	88	69	21	75.6%
Claude 3.5 Sonnet	100	78	72	65	59	74.7%
Inception Mercury 2	100	97	59	54	54	72.8%
Arcee AI: Trinity Large (Preview)	98	87	83	48	46	72.1%
Hermes 3 70B	100	87	67	57	45	71.2%
Llama 3.1 70B	95	68	64	57	47	66.2%
Nemotron 3 Super	100	97	66	46	10	63.8%
Gemini 2.5 Flash (Reasoning)	95	66	59	52	41	62.5%
Gemini 2.5 Flash	100	69	69	63	3	60.8%
Gemini 2.5 Pro	77	63	56	55	37	57.6%
Gemini 2.5 Flash Lite	100	100	65	23	0	57.5%
Gemini 2.5 Flash Lite (Reasoning)	100	90	43	19	10	52.4%
Hermes 3 405B	84	83	47	41	5	52.0%
Mistral Small 3.2 24B	81	72	68	11	1	46.5%
Inception Mercury	82	68	41	29	0	44.0%

Paragraph length variance

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets