Gibberish response detection

Test: Bad Writing Habits

Avg. Score

98.6%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Ministral 3B	100.0%	$0.0001	8.1s	100%
2	Arcee AI: Trinity Mini	100.0%	$0.0003	9.2s	100%
3	Mistral Small Creative	100.0%	$0.0007	9.1s	100%
4	Ministral 8B	100.0%	$0.0004	10.4s	100%
5	Ministral 3 3B	100.0%	$0.0005	11.1s	100%
6	Ministral 3 14B	100.0%	$0.0007	11.7s	100%
7	Mistral NeMO	100.0%	$0.0005	10.1s	99%
8	Gemini 2.5 Flash Lite	100.0%	$0.0009	9.5s	99%
9	Stealth: Aurora Alpha	99.9%	$0.0000	9.8s	99%
10	Claude 3.5 Haiku	100.0%	$0.0035	10.8s	100%
11	GPT-4.1 Nano	99.9%	$0.0007	13.3s	99%
12	Claude 3 Haiku	100.0%	$0.0025	14.9s	100%
13	Gemma 3 4B	100.0%	$0.0002	20.0s	100%
14	Ministral 3 8B	100.0%	$0.0008	19.6s	100%
15	GPT-4.1 Mini	100.0%	$0.0027	19.0s	100%
16	Grok 4 Fast	100.0%	$0.0017	24.1s	100%
17	Gemini 3 Flash (Preview)	100.0%	$0.0078	19.6s	99%
18	Mistral Large 3	100.0%	$0.0033	30.3s	100%
19	GPT-4o Mini (temp=0)	100.0%	$0.0012	34.8s	100%
20	GPT-4o Mini (temp=1)	100.0%	$0.0012	34.8s	100%
21	Claude Haiku 4.5	100.0%	$0.011	21.6s	100%
22	ByteDance Seed 1.6 Flash	99.6%	$0.0013	27.3s	97%
23	Writer: Palmyra X5	100.0%	$0.011	22.0s	100%
24	Grok 4.1 Fast	100.0%	$0.0018	37.8s	100%
25	Gemma 3 12B	100.0%	$0.0004	41.3s	100%
26	Arcee AI: Trinity Large (Preview)	100.0%	$0.0000	43.6s	100%
27	Mistral Medium 3.1	100.0%	$0.0048	36.5s	100%
28	o4 Mini	100.0%	$0.015	25.7s	100%
29	Mistral Large 2	100.0%	$0.013	29.4s	100%
30	Qwen 3.5 Plus (2026-02-15)	99.6%	$0.0060	31.5s	98%
31	Llama 3.1 Nemotron 70B	99.7%	$0.0038	31.7s	97%
32	Mistral Large	100.0%	$0.014	30.9s	100%
33	Gemma 3 27B	100.0%	$0.0006	52.6s	100%
34	DeepSeek-V2 Chat	100.0%	$0.0021	53.3s	100%
35	DeepSeek V3 (2024-12-26)	100.0%	$0.0021	54.6s	100%
36	Hermes 3 405B	100.0%	$0.0032	53.2s	100%
37	GPT-4o, Aug. 6th (temp=0)	100.0%	$0.023	22.7s	100%
38	Gemini 2.5 Flash	99.1%	$0.0052	10.6s	89%
39	GPT-4o, Aug. 6th (temp=1)	99.8%	$0.018	24.4s	97%
40	GPT-4o, May 13th (temp=1)	100.0%	$0.033	14.4s	100%
41	GPT-4o, May 13th (temp=0)	100.0%	$0.035	14.1s	100%
42	GPT-5 Mini	100.0%	$0.0100	57.4s	100%
43	Z.AI GLM 4.7 Flash	100.0%	$0.0017	1.2m	100%
44	GPT-4.1	100.0%	$0.018	44.7s	100%
45	Cohere Command R+ (Aug. 2024)	100.0%	$0.020	52.5s	100%
46	Z.AI GLM 5	100.0%	$0.0084	1.2m	100%
47	o4 Mini High	100.0%	$0.025	47.2s	100%
48	Claude Sonnet 4.6	100.0%	$0.031	39.3s	100%
49	Minimax M2.5	99.5%	$0.0034	1.3m	98%
50	GPT-5 Nano	100.0%	$0.0042	1.4m	100%
51	Claude Sonnet 4.5	100.0%	$0.035	38.1s	100%
52	Gemini 2.5 Pro	100.0%	$0.036	36.2s	100%
53	Claude Sonnet 4	100.0%	$0.032	43.7s	100%
54	Z.AI GLM 4.7	100.0%	$0.010	1.4m	99%
55	Claude 3.5 Sonnet	100.0%	$0.048	35.5s	100%
56	Claude 3.7 Sonnet	100.0%	$0.042	46.7s	100%
57	DeepSeek V3.2	100.0%	$0.0014	1.9m	100%
58	DeepSeek V3.1	99.9%	$0.0020	1.8m	98%
59	Qwen 2.5 72B	98.3%	$0.0010	36.7s	78%
60	Z.AI GLM 4.5	98.8%	$0.0051	42.1s	79%
61	Z.AI GLM 4.6	98.9%	$0.0065	51.5s	79%
62	Gemini 3 Pro (Preview)	100.0%	$0.055	54.4s	100%
63	Claude Opus 4.5	100.0%	$0.070	53.4s	100%
64	ByteDance Seed 1.6	100.0%	$0.013	2.5m	100%
65	Hermes 3 70B	97.4%	$0.0010	1.2m	75%
66	Grok 4	100.0%	$0.048	1.7m	100%
67	GPT-5.2	100.0%	$0.056	1.5m	100%
68	DeepSeek V3 (2025-03-24)	94.6%	$0.0014	39.4s	65%
69	GPT-5.1	100.0%	$0.054	1.8m	100%
70	Claude Opus 4.6	100.0%	$0.078	1.2m	100%
71	Qwen 3.5 397B A17B	99.9%	$0.014	3.0m	99%
72	MoonshotAI: Kimi K2.5	100.0%	$0.019	3.2m	100%
73	Rocinante 12B	89.9%	$0.0014	38.4s	51%
74	GPT-5	100.0%	$0.065	2.8m	100%
75	WizardLM 2 8x22b	95.3%	$0.0026	1.8m	59%
76	Gemini 3.1 Pro (Preview)	100.0%	$0.107	1.8m	100%
77	Llama 3.1 70B	87.2%	$0.0015	29.4s	35%
78	Claude Opus 4	100.0%	$0.209	1.4m	100%
79	Mistral Small 3.2 24B	98.7%	$0.0069	5.7m	79%
80	Llama 3.1 8B	34.4%	$0.0003	1.3m	0%
98.63%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	98	99.6%
Gemini 3 Flash (Preview)	100	100	100	100	96	99.2%
Minimax M2.5	100	100	100	100	96	99.1%
Rocinante 12B	100	100	100	97	96	98.8%
DeepSeek V3 (2025-03-24)	100	100	100	100	28	85.6%
Gemini 2.5 Flash	100	100	90	68	61	83.6%
Llama 3.1 70B	100	100	100	100	0	80.0%
Hermes 3 70B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	0	0	0	40.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	99	99.7%
Minimax M2.5	100	100	100	99	98	99.3%
GPT-4.1 Nano	100	100	100	100	95	99.0%
Stealth: Aurora Alpha	100	100	100	100	94	98.8%
Hermes 3 70B	100	100	100	100	90	97.9%
Mistral Small 3.2 24B	100	100	100	100	85	97.1%
DeepSeek V3 (2025-03-24)	100	100	100	100	38	87.6%
Llama 3.1 70B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	39	0	0	47.8%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	94	98.7%
Hermes 3 70B	100	100	100	95	89	96.8%
Rocinante 12B	100	100	100	92	54	89.2%
Llama 3.1 70B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	15	0	0	0	23.1%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	99.9%
Hermes 3 70B	100	100	100	100	95	99.0%
ByteDance Seed 1.6 Flash	100	100	98	98	98	98.9%
DeepSeek V3 (2025-03-24)	100	100	100	100	64	92.9%
Rocinante 12B	97	92	88	19	0	59.2%
Llama 3.1 8B	100	100	47	0	0	49.5%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	99	99.9%
GPT-4.1	100	100	100	100	98	99.5%
DeepSeek V3 (2025-03-24)	100	100	100	100	93	98.6%
Llama 3.1 70B	100	100	100	100	0	80.0%
Rocinante 12B	100	100	100	85	7	78.4%
Llama 3.1 8B	100	0	0	0	0	20.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	99.9%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	98	99.7%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	84	96.8%
Rocinante 12B	100	98	98	95	93	96.8%
Hermes 3 70B	100	100	100	90	88	95.6%
Llama 3.1 8B	100	0	0	0	0	20.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	97	99.4%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	96	99.2%
Minimax M2.5	100	100	98	98	98	98.7%
DeepSeek V3.1	100	100	100	100	90	98.1%
ByteDance Seed 1.6 Flash	100	100	99	99	91	97.8%
Qwen 2.5 72B	100	100	100	100	63	92.6%
DeepSeek V3 (2025-03-24)	100	100	100	100	54	90.8%
Rocinante 12B	100	100	91	85	73	90.0%
Hermes 3 70B	100	100	100	100	33	86.7%
Z.AI GLM 4.6	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	0	0	0	40.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	99.9%
Minimax M2.5	100	100	100	100	99	99.7%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	99	99.7%
Z.AI GLM 4.7	100	100	100	100	98	99.6%
Llama 3.1 Nemotron 70B	100	100	100	100	92	98.4%
Qwen 2.5 72B	100	100	100	94	0	78.8%
Llama 3.1 8B	100	82	0	0	0	36.3%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	99	99.9%
GPT-5 Nano	100	100	100	100	99	99.7%
Qwen 3.5 Plus (2026-02-15)	100	100	100	96	94	98.2%
Rocinante 12B	100	100	100	96	93	97.7%
DeepSeek V3 (2025-03-24)	100	100	100	95	61	91.1%
Z.AI GLM 4.5	100	100	100	100	0	80.0%
WizardLM 2 8x22b	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	62	0	0	52.5%
Llama 3.1 8B	100	100	30	0	0	46.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	96	99.3%
ByteDance Seed 1.6 Flash	100	100	100	98	97	99.1%
Minimax M2.5	100	100	100	98	97	99.0%
Qwen 2.5 72B	100	100	100	100	92	98.3%
Rocinante 12B	100	100	100	91	44	87.1%
Llama 3.1 8B	100	100	3	0	0	40.6%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	99	99.8%
Gemini 2.5 Flash	100	100	100	100	99	99.8%
Qwen 3.5 397B A17B	100	100	100	100	98	99.5%
ByteDance Seed 1.6 Flash	100	100	100	100	98	99.5%
Qwen 2.5 72B	100	100	100	100	97	99.3%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	96	99.3%
DeepSeek V3 (2025-03-24)	100	100	100	100	34	86.9%
Rocinante 12B	100	100	100	82	27	81.7%
Llama 3.1 70B	100	100	100	100	0	80.0%
Llama 3.1 8B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	98	99.7%
DeepSeek V3 (2025-03-24)	100	100	100	100	95	99.1%
Minimax M2.5	100	100	98	97	96	98.3%
Rocinante 12B	100	100	100	42	0	68.4%
WizardLM 2 8x22b	100	73	0	0	0	34.7%
Llama 3.1 8B	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	99	99	99.6%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	98	99.5%
Minimax M2.5	100	100	100	100	95	98.9%
Llama 3.1 70B	100	100	100	100	84	96.9%
DeepSeek V3 (2025-03-24)	100	100	100	100	56	91.2%
Rocinante 12B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	13	0	0	0	22.5%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	99	99	99.4%
Minimax M2.5	100	100	100	100	97	99.4%
Hermes 3 70B	100	100	100	96	91	97.3%
Llama 3.1 Nemotron 70B	100	100	100	100	85	97.1%
Rocinante 12B	100	100	100	100	61	92.2%
Llama 3.1 8B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	99	99.8%
Grok 4.1 Fast	100	100	100	100	99	99.8%
DeepSeek V3 (2025-03-24)	100	100	100	100	96	99.1%
Llama 3.1 70B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	83	0	0	56.5%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	99.9%
Rocinante 12B	100	100	100	100	96	99.3%
Gemini 2.5 Flash Lite	100	100	100	100	96	99.2%
DeepSeek V3 (2025-03-24)	100	100	100	100	0	80.0%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	100	17	8	65.0%
Llama 3.1 70B	100	100	100	0	0	60.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	98	99.6%
Mistral NeMO	100	100	100	100	97	99.5%
DeepSeek V3 (2025-03-24)	100	100	100	100	97	99.5%
Minimax M2.5	100	100	100	100	97	99.4%
Llama 3.1 70B	100	100	100	100	5	81.0%
Llama 3.1 8B	100	100	100	62	0	72.3%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	99	99.8%
Minimax M2.5	100	100	100	98	97	99.1%
ByteDance Seed 1.6 Flash	100	100	100	100	94	98.7%
Llama 3.1 8B	100	100	0	0	0	40.0%

Gibberish response detection

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets