Overuse of "that" (subordinate clause padding)

Test: Bad Writing Habits

Avg. Score

97.9%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Gemini 2.5 Flash Lite	100.0%	$0.0009	9.5s	100%
2	Arcee AI: Trinity Mini	100.0%	$0.0003	9.2s	99%
3	Mistral Small Creative	99.8%	$0.0007	9.1s	97%
4	GPT-4.1 Mini	99.9%	$0.0027	19.0s	99%
5	Mistral Large 3	100.0%	$0.0033	30.3s	100%
6	GPT-4o Mini (temp=0)	100.0%	$0.0012	34.8s	100%
7	Gemma 3 4B	99.8%	$0.0002	20.0s	96%
8	Writer: Palmyra X5	100.0%	$0.011	22.0s	100%
9	Gemma 3 12B	100.0%	$0.0004	41.3s	100%
10	Grok 4.1 Fast	100.0%	$0.0018	37.8s	100%
11	Mistral Medium 3.1	100.0%	$0.0048	36.5s	100%
12	o4 Mini	100.0%	$0.015	25.7s	100%
13	Grok 4 Fast	99.7%	$0.0017	24.1s	96%
14	Mistral Large	100.0%	$0.014	30.9s	100%
15	Gemma 3 27B	100.0%	$0.0006	52.6s	100%
16	Qwen 3.5 Plus (2026-02-15)	99.8%	$0.0060	31.5s	98%
17	DeepSeek V3 (2025-03-24)	99.8%	$0.0014	39.4s	98%
18	ByteDance Seed 1.6 Flash	99.5%	$0.0013	27.3s	94%
19	DeepSeek-V2 Chat	99.9%	$0.0021	53.3s	99%
20	Ministral 3 3B	99.4%	$0.0005	11.1s	89%
21	Ministral 3B	99.2%	$0.0001	8.1s	88%
22	GPT-4.1	100.0%	$0.018	44.7s	100%
23	DeepSeek V3 (2024-12-26)	99.7%	$0.0021	54.6s	94%
24	Mistral Large 2	99.6%	$0.013	29.4s	93%
25	o4 Mini High	100.0%	$0.025	47.2s	100%
26	GPT-4o, May 13th (temp=1)	99.8%	$0.033	14.4s	96%
27	Gemini 2.5 Pro	100.0%	$0.036	36.2s	100%
28	Z.AI GLM 4.6	99.6%	$0.0065	51.5s	94%
29	Gemini 3 Flash (Preview)	98.9%	$0.0078	19.6s	89%
30	GPT-4o, Aug. 6th (temp=1)	99.4%	$0.018	24.4s	91%
31	GPT-5 Mini	99.6%	$0.0100	57.4s	93%
32	Claude 3.7 Sonnet	100.0%	$0.042	46.7s	100%
33	Z.AI GLM 4.7	99.9%	$0.010	1.4m	98%
34	DeepSeek V3.2	100.0%	$0.0014	1.9m	100%
35	Z.AI GLM 4.7 Flash	99.3%	$0.0017	1.2m	92%
36	DeepSeek V3.1	99.9%	$0.0020	1.8m	98%
37	Qwen 2.5 72B	98.8%	$0.0010	36.7s	84%
38	GPT-4o Mini (temp=1)	98.8%	$0.0012	34.8s	83%
39	Z.AI GLM 4.5	98.4%	$0.0051	42.1s	87%
40	GPT-4.1 Nano	97.4%	$0.0007	13.3s	81%
41	Gemini 3 Pro (Preview)	100.0%	$0.055	54.4s	100%
42	Ministral 8B	97.9%	$0.0004	10.4s	77%
43	Ministral 3 14B	97.7%	$0.0007	11.7s	74%
44	Minimax M2.5	98.4%	$0.0034	1.3m	86%
45	Z.AI GLM 5	98.8%	$0.0084	1.2m	85%
46	Gemini 2.5 Flash	97.8%	$0.0052	10.6s	73%
47	Llama 3.1 Nemotron 70B	97.1%	$0.0038	31.7s	78%
48	Grok 4	100.0%	$0.048	1.7m	100%
49	GPT-5.2	100.0%	$0.056	1.5m	100%
50	Stealth: Aurora Alpha	97.4%	$0.0000	9.8s	70%
51	Claude 3 Haiku	96.1%	$0.0025	14.9s	75%
52	Claude Sonnet 4.5	98.5%	$0.035	38.1s	84%
53	GPT-5.1	100.0%	$0.054	1.8m	100%
54	Mistral NeMO	96.4%	$0.0005	10.1s	69%
55	Ministral 3 8B	96.2%	$0.0008	19.6s	68%
56	Claude Opus 4.5	99.1%	$0.070	53.4s	90%
57	Claude Sonnet 4	97.3%	$0.032	43.7s	80%
58	Qwen 3.5 397B A17B	99.7%	$0.014	3.0m	95%
59	Arcee AI: Trinity Large (Preview)	96.3%	$0.0000	43.6s	70%
60	Claude 3.5 Sonnet	98.0%	$0.048	35.5s	78%
61	Hermes 3 405B	96.7%	$0.0032	53.2s	70%
62	Cohere Command R+ (Aug. 2024)	97.1%	$0.020	52.5s	72%
63	WizardLM 2 8x22b	97.0%	$0.0026	1.8m	76%
64	GPT-5	100.0%	$0.065	2.8m	100%
65	GPT-4o, May 13th (temp=0)	95.8%	$0.035	14.1s	66%
66	GPT-4o, Aug. 6th (temp=0)	95.0%	$0.023	22.7s	64%
67	Hermes 3 70B	95.3%	$0.0010	1.2m	66%
68	Claude Haiku 4.5	93.9%	$0.011	21.6s	63%
69	Rocinante 12B	94.2%	$0.0014	38.4s	61%
70	Claude Opus 4.6	98.1%	$0.078	1.2m	83%
71	Claude Sonnet 4.6	94.5%	$0.031	39.3s	67%
72	Gemini 3.1 Pro (Preview)	99.5%	$0.107	1.8m	91%
73	MoonshotAI: Kimi K2.5	98.6%	$0.019	3.2m	81%
74	Claude 3.5 Haiku	91.9%	$0.0035	10.8s	56%
75	Llama 3.1 8B	92.6%	$0.0003	1.3m	57%
76	GPT-5 Nano	92.4%	$0.0042	1.4m	61%
77	Claude Opus 4	99.8%	$0.209	1.4m	97%
78	Llama 3.1 70B	84.8%	$0.0015	29.4s	36%
79	ByteDance Seed 1.6	88.5%	$0.013	2.5m	47%
80	Mistral Small 3.2 24B	78.0%	$0.0069	5.7m	23%
97.91%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	97	99.4%
Llama 3.1 Nemotron 70B	100	100	100	100	81	96.2%
Cohere Command R+ (Aug. 2024)	100	100	100	100	75	95.1%
Llama 3.1 70B	100	100	100	100	0	80.0%
Mistral NeMO	100	100	100	100	0	80.0%
Mistral Small 3.2 24B	100	100	48	0	0	49.5%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	99	99.7%
Mistral Large 2	100	100	100	100	99	99.7%
Mistral NeMO	100	100	100	100	92	98.4%
Claude Opus 4.6	100	100	100	100	90	98.1%
Llama 3.1 Nemotron 70B	100	100	100	100	87	97.5%
Claude Haiku 4.5	100	100	100	100	87	97.3%
GPT-5 Nano	100	100	100	98	70	93.6%
Claude Sonnet 4.6	100	100	100	100	60	92.1%
Rocinante 12B	100	100	100	100	56	91.3%
Claude 3 Haiku	100	100	100	96	57	90.6%
Qwen 2.5 72B	100	100	100	100	53	90.6%
Llama 3.1 70B	100	100	100	79	56	86.9%
GPT-4o, May 13th (temp=0)	100	100	100	100	0	80.0%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
Hermes 3 70B	100	100	100	83	0	76.7%
Llama 3.1 8B	100	100	100	0	0	60.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	99	99.7%
Llama 3.1 8B	100	100	100	100	97	99.4%
DeepSeek V3 (2025-03-24)	100	100	100	100	93	98.7%
GPT-4.1 Nano	100	100	100	100	83	96.7%
Claude 3 Haiku	100	100	100	100	79	95.8%
Ministral 3 8B	100	100	100	100	78	95.7%
Mistral Small 3.2 24B	100	100	100	100	65	93.0%
Claude Opus 4.5	100	100	100	100	63	92.6%
GPT-5 Nano	100	100	100	100	60	92.0%
Claude Haiku 4.5	100	100	100	100	40	88.0%
Rocinante 12B	100	100	100	82	50	86.5%
ByteDance Seed 1.6	100	100	100	100	31	86.1%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	93	98.5%
Llama 3.1 8B	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	25	8	66.6%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	99	99.9%
Claude Sonnet 4.6	100	100	100	100	96	99.3%
GPT-4.1 Mini	100	100	100	100	94	98.8%
Llama 3.1 8B	100	100	100	100	89	97.8%
Minimax M2.5	100	100	100	100	88	97.5%
GPT-4o, May 13th (temp=1)	100	100	100	100	81	96.2%
ByteDance Seed 1.6	100	100	100	100	69	93.7%
Rocinante 12B	100	100	100	88	77	93.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	73	71	88.8%
Claude 3 Haiku	100	100	100	83	59	88.5%
Llama 3.1 70B	100	100	100	100	41	88.2%
Gemini 2.5 Flash	100	100	100	99	15	82.8%
Cohere Command R+ (Aug. 2024)	100	100	100	100	10	82.1%
Mistral NeMO	100	100	100	69	37	81.1%
Claude Haiku 4.5	100	100	100	48	47	78.8%
Claude 3.5 Haiku	100	100	52	43	28	64.5%
Mistral Small 3.2 24B	100	94	53	0	0	49.4%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	97	99.4%
Llama 3.1 70B	100	100	100	100	96	99.1%
Claude Haiku 4.5	100	100	100	100	95	98.9%
Llama 3.1 Nemotron 70B	100	100	100	100	85	97.1%
GPT-4.1 Nano	100	100	100	100	67	93.4%
Llama 3.1 8B	100	100	100	100	56	91.1%
Mistral Small 3.2 24B	100	100	34	0	0	46.9%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	97	99.4%
Z.AI GLM 5	100	100	100	100	93	98.7%
Hermes 3 70B	100	100	100	100	93	98.7%
Claude Opus 4.6	100	100	100	100	91	98.3%
Claude Sonnet 4.5	100	100	100	100	90	98.0%
Mistral Small Creative	100	100	100	100	85	97.1%
Llama 3.1 Nemotron 70B	100	100	100	100	83	96.7%
Claude Sonnet 4	100	100	100	100	63	92.5%
Z.AI GLM 4.5	100	100	100	100	63	92.5%
GPT-4.1 Nano	100	100	100	100	63	92.5%
WizardLM 2 8x22b	100	100	100	100	57	91.5%
Claude 3.5 Sonnet	100	100	100	96	61	91.3%
Minimax M2.5	100	100	100	100	47	89.3%
GPT-5 Nano	100	100	100	93	47	88.0%
Qwen 2.5 72B	100	100	100	97	42	87.8%
ByteDance Seed 1.6	100	100	100	100	28	85.6%
Mistral NeMO	100	100	100	100	20	83.9%
GPT-4o, Aug. 6th (temp=0)	100	100	100	20	10	66.0%
Llama 3.1 70B	100	100	85	19	0	60.8%
Mistral Small 3.2 24B	100	100	0	0	0	40.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	96	99.3%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	96	99.1%
Llama 3.1 70B	100	100	100	100	96	99.1%
Cohere Command R+ (Aug. 2024)	100	100	100	100	94	98.8%
Minimax M2.5	100	100	100	100	93	98.5%
DeepSeek V3.1	100	100	100	100	91	98.2%
Claude Opus 4.6	100	100	100	100	87	97.5%
Z.AI GLM 4.7 Flash	100	100	100	100	87	97.5%
Claude Opus 4	100	100	100	100	86	97.3%
Llama 3.1 Nemotron 70B	100	100	100	100	85	97.1%
Qwen 3.5 397B A17B	100	100	100	100	75	95.0%
Claude Opus 4.5	100	100	100	100	75	94.9%
Hermes 3 70B	100	100	100	100	73	94.6%
Claude Sonnet 4.6	100	100	100	87	83	94.1%
GPT-4.1 Nano	100	100	100	100	67	93.4%
Claude 3 Haiku	100	100	100	100	61	92.2%
Claude 3.5 Sonnet	100	100	100	100	56	91.1%
Arcee AI: Trinity Large (Preview)	100	100	100	100	41	88.2%
GPT-4o Mini (temp=1)	100	100	100	100	22	84.3%
MoonshotAI: Kimi K2.5	100	100	100	100	15	83.0%
WizardLM 2 8x22b	100	100	100	63	52	82.8%
Hermes 3 405B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	100	61	22	76.5%
GPT-4o, May 13th (temp=0)	100	100	68	52	45	72.9%
GPT-5 Nano	100	100	81	45	0	65.1%
ByteDance Seed 1.6	100	100	83	15	0	59.7%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	98	99.6%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	94	98.8%
Cohere Command R+ (Aug. 2024)	100	100	100	100	92	98.4%
MoonshotAI: Kimi K2.5	100	100	100	100	91	98.2%
Ministral 3B	100	100	100	100	87	97.3%
Grok 4 Fast	100	100	100	100	86	97.2%
Ministral 8B	100	100	100	100	83	96.7%
GPT-4o Mini (temp=1)	100	100	100	92	91	96.5%
Gemma 3 4B	100	100	100	100	82	96.5%
Hermes 3 70B	100	100	100	100	69	93.7%
ByteDance Seed 1.6	100	100	100	98	70	93.6%
Minimax M2.5	100	100	100	94	70	92.9%
Stealth: Aurora Alpha	100	100	100	100	63	92.7%
Llama 3.1 8B	100	100	100	100	50	89.9%
Claude Sonnet 4.6	100	100	100	96	41	87.4%
Arcee AI: Trinity Large (Preview)	100	100	100	100	28	85.6%
WizardLM 2 8x22b	100	100	100	86	33	83.9%
GPT-5 Nano	100	96	83	73	67	83.6%
Claude Opus 4.6	100	100	100	86	27	82.6%
Z.AI GLM 5	100	100	91	87	29	81.6%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	91	5	79.3%
Claude Haiku 4.5	100	100	100	89	0	77.8%
Claude 3 Haiku	100	100	100	54	30	76.7%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	97	99.4%
Z.AI GLM 4.5	100	100	100	100	95	99.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	94	98.8%
Grok 4 Fast	100	100	100	100	85	97.1%
Gemini 3 Flash (Preview)	100	100	100	100	75	95.0%
Claude 3.5 Haiku	100	100	100	100	71	94.3%
Claude Haiku 4.5	100	100	100	100	59	91.8%
GPT-4.1 Nano	100	100	100	100	56	91.1%
GPT-5 Nano	100	100	100	98	56	90.8%
Claude Sonnet 4.6	100	100	100	100	38	87.7%
ByteDance Seed 1.6	100	100	100	100	20	83.9%
Stealth: Aurora Alpha	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	100	0	80.0%
Hermes 3 70B	100	100	100	71	0	74.3%
Mistral Small 3.2 24B	100	100	91	63	15	73.7%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	97	99.4%
Qwen 3.5 397B A17B	100	100	100	100	97	99.3%
Minimax M2.5	100	100	100	100	96	99.1%
Llama 3.1 8B	100	100	100	100	96	99.1%
MoonshotAI: Kimi K2.5	100	100	100	100	95	99.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	92	98.4%
Claude Opus 4.5	100	100	100	100	90	98.1%
Z.AI GLM 4.6	100	100	100	100	85	97.1%
Z.AI GLM 4.5	100	100	100	100	77	95.3%
Claude Opus 4.6	100	100	100	95	81	95.3%
Cohere Command R+ (Aug. 2024)	100	100	100	100	74	94.8%
DeepSeek V3 (2024-12-26)	100	100	100	100	73	94.6%
GPT-5 Mini	100	100	100	100	68	93.6%
Mistral Large 2	100	100	100	100	66	93.1%
ByteDance Seed 1.6 Flash	100	100	100	80	78	91.8%
Mistral NeMO	100	100	100	100	58	91.6%
Gemini 3.1 Pro (Preview)	100	100	100	100	57	91.5%
GPT-4.1 Nano	100	100	100	100	57	91.5%
Z.AI GLM 4.7 Flash	100	100	100	84	70	90.8%
Llama 3.1 70B	100	100	100	100	48	89.5%
Claude Sonnet 4	100	100	100	100	45	88.9%
GPT-4o, Aug. 6th (temp=1)	100	100	100	86	57	88.7%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	38	87.7%
WizardLM 2 8x22b	100	100	100	100	36	87.2%
Claude Sonnet 4.5	100	100	100	100	31	86.1%
Gemini 3 Flash (Preview)	100	100	80	74	69	84.6%
Claude Sonnet 4.6	100	100	100	80	38	83.6%
Stealth: Aurora Alpha	100	100	100	100	0	80.0%
Claude Haiku 4.5	100	100	100	100	0	80.0%
Llama 3.1 Nemotron 70B	100	100	100	85	8	78.7%
Rocinante 12B	100	100	100	92	0	78.4%
Arcee AI: Trinity Large (Preview)	100	100	100	53	0	70.7%
Ministral 8B	100	98	70	63	0	66.3%
ByteDance Seed 1.6	100	100	83	17	13	62.8%
Ministral 3 14B	100	100	68	37	0	60.9%
Ministral 3 8B	100	100	77	24	0	60.0%
Mistral Small 3.2 24B	83	82	71	8	0	48.7%
Claude 3.5 Haiku	100	56	48	8	0	42.2%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	99	99.7%
Z.AI GLM 5	100	100	100	100	96	99.1%
Minimax M2.5	100	100	100	100	89	97.8%
Llama 3.1 Nemotron 70B	100	100	100	93	83	95.2%
GPT-4.1 Nano	100	100	100	100	73	94.6%
Hermes 3 405B	100	100	100	100	61	92.3%
Z.AI GLM 4.5	100	100	100	100	59	91.8%
Claude 3.5 Haiku	100	100	100	81	69	90.0%
Ministral 3 3B	100	100	100	100	48	89.5%
Claude 3.5 Sonnet	100	100	100	100	13	82.6%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	100	0	80.0%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
GPT-5 Nano	100	100	100	70	9	75.8%
Claude Sonnet 4.6	100	100	58	35	18	62.2%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	93	98.5%
Z.AI GLM 4.5	100	100	100	100	91	98.2%
Claude Sonnet 4.5	100	100	100	100	83	96.7%
Llama 3.1 Nemotron 70B	100	100	100	100	77	95.3%
Llama 3.1 8B	100	100	100	79	79	91.6%
GPT-4o, Aug. 6th (temp=0)	100	100	100	77	71	89.6%
Ministral 3B	100	100	100	100	45	89.1%
Llama 3.1 70B	100	100	100	74	25	79.8%
Rocinante 12B	100	100	100	84	0	76.8%
GPT-4o, May 13th (temp=0)	100	100	100	66	0	73.1%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	99	99.9%
Claude 3.5 Sonnet	100	100	100	100	94	98.8%
Ministral 3B	100	100	100	100	94	98.8%
DeepSeek V3 (2025-03-24)	100	100	100	100	91	98.2%
Claude Sonnet 4	100	100	100	100	83	96.7%
Llama 3.1 8B	100	100	100	100	83	96.7%
ByteDance Seed 1.6	100	100	100	100	78	95.6%
Claude Sonnet 4.6	100	100	100	100	77	95.3%
Rocinante 12B	100	100	100	87	64	90.3%
Hermes 3 405B	100	100	100	100	48	89.5%
Llama 3.1 70B	100	100	100	97	0	79.4%
GPT-5 Nano	100	100	100	60	32	78.3%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	99	99.9%
Claude Haiku 4.5	100	100	100	100	99	99.9%
DeepSeek V3.1	100	100	100	100	99	99.9%
Llama 3.1 70B	100	100	100	100	97	99.4%
GPT-5 Nano	100	100	100	100	84	96.8%
Hermes 3 405B	100	100	100	100	79	95.8%
Claude Opus 4.6	100	100	100	100	74	94.8%
Arcee AI: Trinity Large (Preview)	100	100	100	97	77	94.8%
Hermes 3 70B	100	100	100	100	52	90.3%
Claude Sonnet 4	100	100	100	100	42	88.3%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
ByteDance Seed 1.6	100	100	100	28	22	69.9%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	94	98.8%
Llama 3.1 8B	100	100	100	97	83	96.1%
Claude Sonnet 4	100	100	100	100	77	95.3%
Z.AI GLM 4.5	100	100	100	100	77	95.3%
Claude 3.5 Haiku	100	100	100	100	74	94.8%
Claude Haiku 4.5	100	100	100	82	74	91.3%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	98	99.5%
Claude Sonnet 4.5	100	100	100	100	93	98.7%
Gemini 2.5 Flash	100	100	100	100	90	98.0%
GPT-4o Mini (temp=1)	100	100	100	100	90	98.0%
Z.AI GLM 4.7	100	100	100	100	89	97.7%
Llama 3.1 Nemotron 70B	100	100	100	100	87	97.5%
Ministral 3 14B	100	100	100	100	87	97.3%
Minimax M2.5	100	100	100	100	82	96.3%
Arcee AI: Trinity Large (Preview)	100	100	100	95	86	96.3%
Z.AI GLM 4.6	100	100	100	100	75	95.0%
MoonshotAI: Kimi K2.5	100	100	100	100	71	94.1%
Claude Haiku 4.5	100	100	96	90	49	86.9%
Claude 3 Haiku	100	100	100	70	63	86.5%
Hermes 3 405B	100	100	100	100	19	83.7%
Mistral Small 3.2 24B	100	100	100	100	13	82.7%
Llama 3.1 70B	100	100	100	100	4	80.8%
Cohere Command R+ (Aug. 2024)	100	100	100	100	3	80.5%
Rocinante 12B	100	100	100	100	0	80.0%
ByteDance Seed 1.6	100	100	100	85	0	77.1%
Ministral 3 8B	100	100	100	50	32	76.3%
Claude 3.5 Haiku	100	100	100	28	15	68.6%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Stealth: Aurora Alpha	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 14B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	96	99.1%
Z.AI GLM 4.5	100	100	100	100	95	99.0%
DeepSeek-V2 Chat	100	100	100	100	94	98.8%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	93	98.7%
GPT-4o, May 13th (temp=0)	100	100	100	100	90	98.0%
Llama 3.1 Nemotron 70B	100	100	100	100	83	96.7%
Claude Sonnet 4.5	100	100	100	100	69	93.7%
Claude Sonnet 4	100	100	100	84	70	90.9%
Llama 3.1 8B	100	100	97	96	56	89.7%
Hermes 3 70B	100	100	100	94	43	87.5%
ByteDance Seed 1.6	100	100	100	66	59	85.0%
Gemini 2.5 Flash	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	93	0	78.5%

Overuse of "that" (subordinate clause padding)

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets