Useless dialogue additions

Test: Bad Writing Habits

Avg. Score

51.7%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	GPT-5.4 Mini	87.2%	$0.015	16.8s	54%
2	Claude Sonnet 4.6	92.4%	$0.031	39.3s	60%
3	Qwen 3.5 35B	93.3%	$0.018	1.0m	58%
4	GPT-5.4 Mini (Reasoning)	90.0%	$0.022	28.1s	55%
5	GPT-5.4	93.4%	$0.049	1.4m	68%
6	GPT-5.4 Mini (Reasoning, Low)	84.0%	$0.015	16.8s	47%
7	Qwen 3.5 9B	89.3%	$0.0011	1.4m	48%
8	Z.AI GLM 5 Turbo	84.3%	$0.0081	33.2s	41%
9	Qwen 3.5 397B A17B	96.4%	$0.014	3.0m	64%
10	Claude Sonnet 4.6 (Reasoning)	90.8%	$0.060	1.2m	59%
11	GPT-5.4 (Reasoning, Low)	91.0%	$0.055	1.4m	58%
12	Claude Haiku 4.5	79.8%	$0.011	21.6s	38%
13	MiniMax M2.5	80.7%	$0.0034	1.3m	44%
14	Qwen 3.5 Flash	84.5%	$0.0025	47.5s	35%
15	Claude Opus 4.6	90.2%	$0.078	1.2m	56%
16	DeepSeek V3 (2025-03-24)	79.4%	$0.0014	39.4s	32%
17	Claude Opus 4.6 (Reasoning)	92.0%	$0.088	1.4m	58%
18	Qwen3 235B A22B Instruct 2507	77.9%	$0.0011	59.2s	36%
19	GPT-5 Mini	76.6%	$0.0100	57.4s	38%
20	Mistral Small 4 (Reasoning)	73.0%	$0.0022	30.2s	32%
21	GPT-5.1	86.9%	$0.054	1.8m	54%
22	Mistral Small 4	71.4%	$0.0014	18.2s	27%
23	GPT-5.4 Nano (Reasoning)	70.8%	$0.0061	24.5s	30%
24	MiniMax M2.7	76.2%	$0.0040	1.1m	34%
25	Claude Sonnet 4.5	78.6%	$0.035	38.1s	35%
26	GPT-5	89.7%	$0.065	2.8m	63%
27	Z.AI GLM 5	77.9%	$0.0084	1.2m	31%
28	Writer: Palmyra X5	67.9%	$0.011	22.0s	28%
29	GPT-5.4 Nano	65.7%	$0.0057	26.3s	27%
30	Qwen 3.5 122B	80.1%	$0.025	1.1m	28%
31	GPT-5.4 Nano (Reasoning, Low)	65.7%	$0.0055	20.6s	23%
32	Qwen 3.5 27B	82.4%	$0.020	1.6m	31%
33	GPT-5.4 (Reasoning)	91.7%	$0.089	2.6m	58%
34	Claude Opus 4.5	77.4%	$0.070	53.4s	40%
35	Grok 4.20 (Beta)	63.0%	$0.018	15.8s	23%
36	Grok 4.20 (Beta, Reasoning)	72.4%	$0.039	34.0s	26%
37	Claude 3.5 Haiku	70.9%	$0.0035	10.8s	10%
38	Claude Sonnet 4	69.2%	$0.032	43.7s	24%
39	o4 Mini	61.8%	$0.015	25.7s	19%
40	Rocinante 12B	67.1%	$0.0014	38.4s	13%
41	Qwen 3 32B	61.6%	$0.0015	54.6s	20%
42	Claude 3.7 Sonnet	65.6%	$0.042	46.7s	26%
43	ByteDance Seed 2.0 Lite	76.8%	$0.012	2.2m	25%
44	Claude 3.5 Sonnet	68.2%	$0.048	35.5s	21%
45	Mistral Medium 3.1	57.0%	$0.0048	36.5s	14%
46	Gemini 3.1 Flash Lite (Preview)	55.7%	$0.0030	8.4s	6%
47	o4 Mini High	62.9%	$0.025	47.2s	15%
48	MoonshotAI: Kimi K2.5	75.9%	$0.019	3.2m	31%
49	Grok 4.1 Fast	52.3%	$0.0018	37.8s	11%
50	Mistral Large 2	55.9%	$0.013	29.4s	9%
51	Gemini 3.1 Pro (Preview)	85.6%	$0.107	1.8m	34%
52	Hermes 3 405B	56.1%	$0.0032	53.2s	10%
53	Llama 3.1 8B	60.0%	$0.0003	1.3m	10%
54	Stealth: Healer Alpha	47.3%	$0.0000	23.7s	7%
55	ByteDance Seed 1.6 Flash	43.9%	$0.0013	27.3s	11%
56	GPT-4.1	51.5%	$0.018	44.7s	14%
57	GPT-5.2	66.4%	$0.056	1.5m	24%
58	Mistral Large 3	50.0%	$0.0033	30.3s	5%
59	LFM2 24B	42.8%	$0.0002	28.4s	7%
60	Mistral Small Creative	41.1%	$0.0007	9.1s	4%
61	Ministral 3 14B	40.6%	$0.0007	11.7s	4%
62	Z.AI GLM 4.5	44.6%	$0.0051	42.1s	9%
63	Mistral Large	48.4%	$0.014	30.9s	5%
64	GPT-4o, Aug. 6th (temp=1)	43.2%	$0.018	24.4s	6%
65	Gemini 2.5 Flash	37.3%	$0.0052	10.6s	2%
66	ByteDance Seed 1.6	62.9%	$0.013	2.5m	14%
67	Grok 4 Fast	34.9%	$0.0017	24.1s	4%
68	Stealth: Hunter Alpha	41.8%	$0.0000	55.0s	5%
69	Gemini 2.5 Flash Lite (Reasoning)	39.0%	$0.0028	30.8s	1%
70	Hermes 3 70B	43.6%	$0.0010	1.2m	5%
71	Aion 2.0	43.2%	$0.0064	1.3m	8%
72	Llama 3.1 70B	36.6%	$0.0015	29.4s	0%
73	Ministral 3B	29.4%	$0.0001	8.1s	0%
74	Z.AI GLM 4.7 Flash	39.1%	$0.0017	1.2m	6%
75	WizardLM 2 8x22b	48.6%	$0.0026	1.8m	7%
76	Ministral 8B	28.1%	$0.0004	10.4s	0%
77	Z.AI GLM 4.7	40.4%	$0.010	1.4m	10%
78	Gemini 2.5 Flash (Reasoning)	34.1%	$0.011	21.5s	1%
79	Claude Opus 4	82.3%	$0.209	1.4m	45%
80	GPT-5 Nano	37.2%	$0.0042	1.4m	8%
81	Arcee AI: Trinity Mini	24.8%	$0.0003	9.2s	0%
82	Gemini 3 Flash (Preview, Reasoning)	34.9%	$0.012	30.1s	1%
83	Llama 3.1 Nemotron 70B	31.5%	$0.0038	31.7s	0%
84	Gemini 2.5 Flash Lite	23.7%	$0.0009	9.5s	0%
85	Z.AI GLM 4.6	38.0%	$0.0065	51.5s	0%
86	GPT-4.1 Nano	24.0%	$0.0007	13.3s	0%
87	Ministral 3 8B	25.5%	$0.0008	19.6s	0%
88	Ministral 3 3B	22.4%	$0.0005	11.1s	0%
89	Gemini 3 Pro (Preview)	44.0%	$0.055	54.4s	10%
90	Arcee AI: Trinity Large (Preview)	29.8%	$0.0000	43.6s	0%
91	Inception Mercury	27.3%	$0.011	17.6s	0%
92	Gemini 2.5 Pro	42.2%	$0.036	36.2s	1%
93	DeepSeek V3 (2024-12-26)	33.1%	$0.0021	54.6s	0%
94	DeepSeek-V2 Chat	32.0%	$0.0021	53.3s	0%
95	Claude 3 Haiku	20.7%	$0.0025	14.9s	0%
96	Gemini 3 Flash (Preview)	23.3%	$0.0078	19.6s	0%
97	GPT-4.1 Mini	20.3%	$0.0027	19.0s	0%
98	DeepSeek V3.2	38.2%	$0.0014	1.9m	5%
99	Qwen 3.5 Plus (2026-02-15)	20.3%	$0.0060	31.5s	0%
100	GPT-4o, May 13th (temp=1)	27.8%	$0.033	14.4s	0%
101	Mistral NeMO	10.9%	$0.0005	10.1s	0%
102	GPT-4o Mini (temp=1)	16.7%	$0.0012	34.8s	0%
103	Cohere Command R+ (Aug. 2024)	25.6%	$0.020	52.5s	0%
104	Gemma 3 27B	15.1%	$0.0006	52.6s	0%
105	Stealth: Aurora Alpha	2.3%	$0.0000	9.8s	0%
106	Inception Mercury 2	1.6%	$0.0032	7.0s	0%
107	Nemotron 3 Super	22.6%	$0.0000	1.4m	0%
108	Gemma 3 4B	3.1%	$0.0002	20.0s	0%
109	DeepSeek V3.1	29.2%	$0.0020	1.8m	0%
110	Nemotron 3 Nano	15.9%	$0.0010	1.1m	0%
111	Gemma 3 12B	7.3%	$0.0004	41.3s	0%
112	Qwen 2.5 72B	3.6%	$0.0010	36.7s	0%
113	GPT-4o, Aug. 6th (temp=0)	7.5%	$0.023	22.7s	0%
114	GPT-4o Mini (temp=0)	0.0%	$0.0012	34.8s	0%
115	GPT-4o, May 13th (temp=0)	8.7%	$0.035	14.1s	0%
116	ByteDance Seed 2.0 Mini	56.7%	$0.0045	4.9m	10%
117	Grok 4	31.3%	$0.048	1.7m	1%
118	Mistral Small 3.2 24B	16.1%	$0.0069	5.7m	0%
51.72%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	97	96	98.6%
GPT-5.4 (Reasoning)	100	100	100	100	92	98.3%
GPT-5.4 Mini	100	100	100	100	89	97.9%
GPT-5.4	100	100	97	94	92	96.5%
GPT-5.1	100	100	100	100	78	95.6%
GPT-5	100	100	100	85	83	93.7%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	87	58	89.0%
Claude Opus 4	100	100	100	83	48	86.3%
Claude Sonnet 4.6 (Reasoning)	100	100	100	75	54	85.7%
Claude Sonnet 4.6	100	100	80	80	63	84.3%
Qwen 3.5 Flash	100	100	100	69	51	84.2%
Qwen3 235B A22B Instruct 2507	100	100	99	92	25	83.1%
Claude Opus 4.6	100	100	92	89	34	83.0%
Claude Opus 4.6 (Reasoning)	100	100	92	87	34	82.5%
GPT-5.2	100	100	98	64	48	82.1%
ByteDance Seed 2.0 Mini	100	100	100	75	25	80.0%
Claude Haiku 4.5	100	100	89	72	37	79.7%
Mistral Small 4 (Reasoning)	100	100	100	54	34	77.5%
Claude 3.5 Sonnet	97	83	72	69	63	77.0%
Rocinante 12B	100	100	100	80	0	75.9%
GPT-5 Mini	100	100	95	66	16	75.3%
Claude Opus 4.5	100	96	71	63	46	75.1%
ByteDance Seed 1.6	100	100	100	69	0	73.9%
Z.AI GLM 5	100	100	69	54	46	73.8%
ByteDance Seed 2.0 Lite	100	100	100	63	0	72.5%
Grok 4.20 (Beta, Reasoning)	100	100	75	66	14	71.0%
Grok 4.20 (Beta)	100	100	100	34	21	71.0%
MiniMax M2.7	100	77	69	54	46	69.3%
Mistral Small 4	96	75	72	54	48	68.9%
Grok 4.1 Fast	100	100	75	69	0	68.9%
DeepSeek-V2 Chat	100	100	80	63	0	68.4%
o4 Mini	100	94	83	63	0	67.9%
Qwen 3 32B	96	96	81	67	0	67.7%
Writer: Palmyra X5	100	75	69	54	37	67.0%
GPT-5.4 Nano (Reasoning)	100	94	66	50	0	62.0%
Claude 3.7 Sonnet	86	82	72	34	28	60.3%
Z.AI GLM 4.7 Flash	100	100	66	25	0	58.2%
DeepSeek V3 (2024-12-26)	100	100	42	42	0	56.7%
DeepSeek V3 (2025-03-24)	100	100	54	25	0	55.7%
WizardLM 2 8x22b	100	97	42	25	14	55.6%
o4 Mini High	100	89	66	14	0	53.9%
GPT-4.1	87	75	69	37	0	53.5%
Z.AI GLM 5 Turbo	96	94	58	0	0	49.5%
GPT-5.4 Nano (Reasoning, Low)	100	84	38	25	0	49.5%
Gemini 2.5 Flash	100	100	37	0	0	47.4%
Z.AI GLM 4.7	87	75	38	31	0	46.2%
MoonshotAI: Kimi K2.5	100	87	42	0	0	45.6%
GPT-5.4 Nano	80	69	54	25	0	45.5%
Grok 4 Fast	100	100	25	0	0	45.0%
Claude Sonnet 4	89	87	31	0	0	41.4%
Claude 3.5 Haiku	100	100	0	0	0	40.0%
Hermes 3 70B	100	100	0	0	0	40.0%
Grok 4	100	75	14	0	0	37.8%
DeepSeek V3.1	100	75	0	0	0	35.0%
Cohere Command R+ (Aug. 2024)	100	54	18	0	0	34.3%
Llama 3.1 8B	100	69	0	0	0	33.9%
Ministral 3 14B	100	54	0	0	0	30.7%
MiniMax M2.5	54	46	46	0	0	29.1%
Ministral 3 8B	100	25	14	0	0	27.8%
Mistral Large 3	100	34	0	0	0	26.8%
Mistral Large	69	34	25	0	0	25.7%
Stealth: Healer Alpha	100	25	0	0	0	25.0%
Gemini 3 Flash (Preview, Reasoning)	72	31	20	0	0	24.7%
Gemini 3 Pro (Preview)	49	32	21	16	0	23.8%
ByteDance Seed 1.6 Flash	51	39	25	0	0	22.9%
Z.AI GLM 4.5	63	49	0	0	0	22.3%
Gemini 2.5 Flash Lite (Reasoning)	100	10	0	0	0	21.9%
Mistral Medium 3.1	63	42	0	0	0	20.8%
Qwen 3.5 Plus (2026-02-15)	38	34	20	8	0	20.0%
Z.AI GLM 4.6	100	0	0	0	0	20.0%
GPT-4o, May 13th (temp=1)	100	0	0	0	0	20.0%
Gemini 3.1 Flash Lite (Preview)	37	25	20	14	0	19.1%
GPT-4o, Aug. 6th (temp=1)	54	42	0	0	0	19.0%
Arcee AI: Trinity Large (Preview)	63	25	5	0	0	18.5%
Hermes 3 405B	80	7	0	0	0	17.4%
Llama 3.1 70B	80	0	0	0	0	15.9%
GPT-5 Nano	67	10	0	0	0	15.4%
Aion 2.0	34	20	0	0	0	10.8%
Llama 3.1 Nemotron 70B	54	0	0	0	0	10.7%
Ministral 3B	54	0	0	0	0	10.7%
GPT-4.1 Mini	25	18	10	0	0	10.5%
GPT-4o Mini (temp=1)	36	10	6	0	0	10.3%
Gemini 3 Flash (Preview)	46	0	0	0	0	9.2%
Claude 3 Haiku	34	10	0	0	0	8.7%
DeepSeek V3.2	25	14	0	0	0	7.8%
Mistral Large 2	34	0	0	0	0	6.8%
Gemma 3 12B	25	0	0	0	0	5.0%
Mistral Small Creative	25	0	0	0	0	5.0%
Ministral 8B	25	0	0	0	0	5.0%
GPT-4.1 Nano	14	0	0	0	0	2.8%
LFM2 24B	14	0	0	0	0	2.8%
Gemini 2.5 Flash (Reasoning)	10	0	0	0	0	1.9%
Gemini 2.5 Pro	0	0	0	0	0	0.0%
Stealth: Hunter Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 27B	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	80	95.9%
GPT-5.1	100	100	100	92	77	93.8%
Grok 4.20 (Beta)	100	100	100	89	69	91.7%
Gemini 3.1 Pro (Preview)	100	100	100	100	54	90.7%
MiniMax M2.7	100	100	100	100	54	90.7%
GPT-5.4 (Reasoning)	100	100	100	100	42	88.3%
GPT-5.4	100	100	100	100	42	88.3%
Claude Opus 4	100	100	99	83	55	87.5%
Gemini 3 Pro (Preview)	100	100	83	80	69	86.5%
Claude 3.5 Sonnet	100	100	100	75	54	85.7%
Arcee AI: Trinity Mini	100	100	100	100	25	85.0%
MiniMax M2.5	100	100	100	69	54	84.6%
Z.AI GLM 5 Turbo	100	100	100	100	0	80.0%
Claude Opus 4.6	100	100	100	100	0	80.0%
Qwen 3.5 122B	100	100	100	100	0	80.0%
Z.AI GLM 5	100	100	100	100	0	80.0%
MoonshotAI: Kimi K2.5	100	100	100	100	0	80.0%
o4 Mini High	100	100	100	100	0	80.0%
Claude Opus 4.5	100	100	100	75	25	80.0%
Aion 2.0	100	100	100	100	0	80.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	0	80.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	0	80.0%
Claude Haiku 4.5	100	100	100	100	0	80.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	0	80.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	0	80.0%
Inception Mercury	100	100	100	100	0	80.0%
Ministral 3 3B	100	100	100	100	0	80.0%
Qwen 3 32B	100	100	83	57	50	78.0%
ByteDance Seed 1.6	100	100	100	80	0	75.9%
Hermes 3 405B	100	100	100	69	0	73.9%
Writer: Palmyra X5	100	100	100	69	0	73.9%
Mistral Small 4	100	100	100	69	0	73.9%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	54	14	73.5%
Llama 3.1 8B	100	100	100	63	0	72.5%
o4 Mini	100	100	100	42	0	68.3%
Claude 3.7 Sonnet	100	100	71	50	14	67.0%
Ministral 8B	100	100	100	25	0	65.0%
GPT-5 Mini	99	89	73	54	0	63.0%
Rocinante 12B	100	100	100	7	0	61.4%
GPT-5.4 (Reasoning, Low)	100	100	63	42	0	60.8%
Grok 4.20 (Beta, Reasoning)	100	100	100	0	0	60.0%
GPT-4.1	100	100	100	0	0	60.0%
Grok 4 Fast	100	100	100	0	0	60.0%
Stealth: Healer Alpha	100	100	100	0	0	60.0%
Mistral Large 3	100	100	100	0	0	60.0%
Mistral Large 2	100	100	100	0	0	60.0%
DeepSeek V3.1	100	100	100	0	0	60.0%
DeepSeek V3.2	100	100	100	0	0	60.0%
Llama 3.1 70B	100	100	100	0	0	60.0%
Mistral Medium 3.1	100	100	100	0	0	60.0%
Mistral Small Creative	100	100	100	0	0	60.0%
GPT-4.1 Nano	100	100	100	0	0	60.0%
WizardLM 2 8x22b	100	100	100	0	0	60.0%
ByteDance Seed 1.6 Flash	100	77	69	48	0	59.0%
GPT-5.2	100	100	54	25	0	55.7%
GPT-5 Nano	83	60	57	42	34	55.1%
GPT-5.4 Nano	100	100	63	0	0	52.5%
Z.AI GLM 4.7	100	100	25	25	0	50.0%
GPT-5.4 Nano (Reasoning)	100	63	42	42	0	49.2%
Grok 4	100	100	42	0	0	48.3%
GPT-4.1 Mini	100	80	54	0	0	46.6%
GPT-4o, Aug. 6th (temp=1)	100	63	42	25	0	45.8%
Grok 4.1 Fast	100	100	25	0	0	45.0%
Ministral 3 14B	100	100	25	0	0	45.0%
LFM2 24B	100	100	25	0	0	45.0%
Qwen 3.5 Plus (2026-02-15)	83	80	34	25	0	44.4%
Stealth: Hunter Alpha	100	100	14	0	0	42.8%
GPT-4o, May 13th (temp=0)	100	100	0	0	0	40.0%
DeepSeek-V2 Chat	100	100	0	0	0	40.0%
DeepSeek V3 (2024-12-26)	100	100	0	0	0	40.0%
Hermes 3 70B	100	100	0	0	0	40.0%
Ministral 3 8B	100	100	0	0	0	40.0%
Claude 3 Haiku	100	100	0	0	0	40.0%
Ministral 3B	100	100	0	0	0	40.0%
Gemini 3 Flash (Preview, Reasoning)	100	75	0	0	0	35.0%
Arcee AI: Trinity Large (Preview)	96	42	25	7	0	33.9%
Gemini 3.1 Flash Lite (Preview)	80	63	25	0	0	33.4%
Gemini 3 Flash (Preview)	42	42	42	42	0	33.3%
Z.AI GLM 4.5	100	54	0	0	0	30.7%
GPT-4o Mini (temp=1)	100	42	0	0	0	28.3%
Z.AI GLM 4.7 Flash	100	25	0	0	0	25.0%
Gemma 3 27B	100	25	0	0	0	25.0%
Nemotron 3 Nano	100	14	0	0	0	22.8%
Nemotron 3 Super	100	0	0	0	0	20.0%
Gemini 2.5 Flash Lite	100	0	0	0	0	20.0%
Gemini 2.5 Flash	100	0	0	0	0	20.0%
Mistral NeMO	100	0	0	0	0	20.0%
GPT-4o, May 13th (temp=1)	42	34	0	0	0	15.2%
Gemma 3 4B	25	25	14	0	0	12.8%
Cohere Command R+ (Aug. 2024)	54	0	0	0	0	10.7%
Llama 3.1 Nemotron 70B	25	0	0	0	0	5.0%
Stealth: Aurora Alpha	18	0	0	0	0	3.6%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	97	99.4%
GPT-5	100	100	100	100	97	99.3%
Qwen3 235B A22B Instruct 2507	100	100	100	100	94	98.8%
GPT-5.2	100	100	100	100	93	98.5%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	92	98.3%
Claude Opus 4.6 (Reasoning)	100	100	100	100	88	97.6%
GPT-5.4 Mini	100	100	100	97	89	97.3%
Z.AI GLM 5 Turbo	100	100	100	94	89	96.6%
Claude Sonnet 4.6 (Reasoning)	100	100	100	94	89	96.6%
Z.AI GLM 5	100	100	100	92	83	95.0%
MiniMax M2.5	100	100	100	89	83	94.5%
GPT-5.4 Mini (Reasoning)	100	100	100	97	72	93.9%
Claude 3.5 Sonnet	100	100	100	100	69	93.9%
Mistral Medium 3.1	100	100	100	100	54	90.7%
Claude Opus 4	100	100	100	83	67	90.1%
Claude Haiku 4.5	100	100	96	91	63	89.7%
MoonshotAI: Kimi K2.5	100	100	97	80	69	89.2%
Claude Opus 4.6	100	100	100	100	34	86.8%
Hermes 3 405B	100	100	100	100	34	86.8%
Mistral Large 3	100	100	97	89	46	86.5%
Claude Sonnet 4	100	100	100	100	31	86.3%
Claude 3.7 Sonnet	100	100	89	76	63	85.6%
Writer: Palmyra X5	100	97	88	69	67	84.4%
Claude Opus 4.5	100	100	100	66	54	83.9%
Mistral Small 4	100	100	100	63	56	83.6%
o4 Mini	100	83	83	80	66	82.5%
Grok 4.20 (Beta, Reasoning)	100	97	83	63	63	81.1%
ByteDance Seed 2.0 Lite	100	100	100	63	42	80.8%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Qwen 3.5 122B	100	100	100	99	0	79.7%
GPT-5.4 Nano	100	100	71	68	50	77.8%
Z.AI GLM 4.7 Flash	100	100	100	82	0	76.3%
GPT-5 Mini	90	88	81	79	42	75.9%
Grok 4.20 (Beta)	100	80	72	67	58	75.4%
Arcee AI: Trinity Large (Preview)	100	100	94	83	0	75.4%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	74	0	74.7%
Gemini 3 Pro (Preview)	96	89	80	73	28	73.3%
Z.AI GLM 4.5	96	85	73	56	54	72.7%
o4 Mini High	100	100	71	54	25	70.0%
Mistral Large 2	99	97	87	66	0	69.7%
ByteDance Seed 1.6 Flash	100	100	91	34	18	68.6%
GPT-5.4 Nano (Reasoning)	100	89	71	54	25	67.9%
Mistral Small 4 (Reasoning)	100	100	58	42	37	67.4%
WizardLM 2 8x22b	100	100	78	50	6	66.8%
ByteDance Seed 1.6	100	100	100	25	0	65.0%
Stealth: Healer Alpha	100	77	63	28	25	58.6%
Ministral 3 14B	100	75	63	42	11	58.1%
Rocinante 12B	100	87	69	34	0	58.0%
Qwen 3 32B	87	80	75	42	0	56.6%
GPT-4o, Aug. 6th (temp=1)	92	80	54	30	25	55.9%
Aion 2.0	92	87	72	11	7	53.9%
Llama 3.1 8B	100	97	60	0	0	51.4%
Z.AI GLM 4.7	77	72	63	42	0	50.6%
Mistral Small Creative	100	75	63	7	0	49.0%
Stealth: Hunter Alpha	80	57	56	50	0	48.4%
GPT-4.1	80	72	66	14	0	46.4%
Ministral 3 8B	91	75	54	7	0	45.3%
Grok 4.1 Fast	75	54	48	42	0	43.7%
LFM2 24B	89	69	50	7	0	43.2%
Mistral Large	83	63	42	18	10	43.0%
DeepSeek V3 (2024-12-26)	96	54	54	0	0	40.5%
Ministral 3B	100	100	0	0	0	40.0%
GPT-5 Nano	100	71	21	4	0	39.3%
Llama 3.1 Nemotron 70B	100	87	0	0	0	37.3%
Claude 3 Haiku	92	89	0	0	0	36.2%
DeepSeek-V2 Chat	100	75	0	0	0	35.0%
Gemma 3 12B	66	50	30	11	0	31.5%
DeepSeek V3.2	66	48	30	11	0	31.1%
Qwen 3.5 Plus (2026-02-15)	56	51	37	5	0	29.7%
Hermes 3 70B	100	48	0	0	0	29.6%
ByteDance Seed 2.0 Mini	75	46	25	0	0	29.2%
Gemini 2.5 Flash	100	44	0	0	0	28.8%
DeepSeek V3.1	100	25	14	0	0	27.8%
Gemini 3 Flash (Preview, Reasoning)	100	25	0	0	0	25.0%
Gemini 2.5 Flash (Reasoning)	51	45	21	0	0	23.5%
GPT-4o Mini (temp=1)	54	36	17	11	0	23.4%
Nemotron 3 Super	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
GPT-4o, May 13th (temp=1)	51	45	0	0	0	19.2%
Grok 4 Fast	38	37	14	6	0	18.9%
Gemma 3 27B	54	38	0	0	0	18.3%
Qwen 2.5 72B	80	0	0	0	0	15.9%
Grok 4	75	4	0	0	0	15.9%
Z.AI GLM 4.6	34	25	14	0	0	14.6%
Gemini 3 Flash (Preview)	31	22	18	0	0	14.1%
GPT-4.1 Mini	69	0	0	0	0	13.9%
Arcee AI: Trinity Mini	63	0	0	0	0	12.5%
Cohere Command R+ (Aug. 2024)	63	0	0	0	0	12.5%
Gemma 3 4B	63	0	0	0	0	12.5%
Mistral NeMO	58	0	0	0	0	11.7%
Ministral 8B	54	0	0	0	0	10.7%
GPT-4o, May 13th (temp=0)	25	25	0	0	0	10.0%
Inception Mercury	47	0	0	0	0	9.3%
Gemini 3.1 Flash Lite (Preview)	37	10	0	0	0	9.3%
Gemini 2.5 Flash Lite	25	0	0	0	0	5.0%
Gemini 2.5 Pro	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	99	99.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	97	99.4%
GPT-4.1	100	100	100	96	96	98.2%
Qwen 3.5 397B A17B	100	100	100	100	83	96.7%
GPT-5.4 Mini	100	100	100	100	83	96.7%
Claude Sonnet 4.6	100	100	100	100	75	95.0%
GPT-5.2	100	100	100	92	83	95.0%
GPT-5.1	100	100	100	100	71	94.3%
Claude Sonnet 4	100	100	100	100	69	93.9%
DeepSeek V3 (2025-03-24)	100	100	100	100	69	93.9%
o4 Mini	100	100	100	92	75	93.3%
ByteDance Seed 2.0 Lite	100	100	100	100	63	92.5%
Grok 4.20 (Beta)	100	100	100	92	69	92.2%
Claude Haiku 4.5	100	100	99	97	58	90.8%
Claude Opus 4.6	100	100	100	83	63	89.2%
Claude 3.7 Sonnet	100	100	89	88	67	88.9%
GPT-5.4 (Reasoning)	100	100	100	100	42	88.3%
GPT-5.4 (Reasoning, Low)	100	100	100	100	42	88.3%
Mistral Small 4 (Reasoning)	100	100	100	100	42	88.3%
GPT-5 Mini	100	100	100	83	58	88.2%
Arcee AI: Trinity Large (Preview)	100	94	89	87	71	88.2%
GPT-5.4 Nano (Reasoning)	100	100	100	72	63	87.0%
GPT-5.4 Nano	100	100	87	77	69	86.7%
Rocinante 12B	100	100	80	75	63	83.4%
Z.AI GLM 5	100	100	99	88	30	83.3%
Claude 3.5 Sonnet	100	100	75	75	66	83.2%
Claude Sonnet 4.5	100	92	92	80	48	82.2%
Mistral Large	100	94	83	75	58	82.1%
Hermes 3 405B	100	100	100	54	54	81.4%
Writer: Palmyra X5	100	100	99	83	25	81.4%
WizardLM 2 8x22b	100	100	100	56	50	81.2%
ByteDance Seed 1.6	100	100	100	100	0	80.0%
Qwen 3.5 Flash	100	100	100	100	0	80.0%
MiniMax M2.7	100	99	89	58	54	80.0%
Stealth: Hunter Alpha	100	100	94	80	20	78.6%
ByteDance Seed 2.0 Mini	100	100	100	42	42	76.7%
Claude Sonnet 4.6 (Reasoning)	100	96	94	66	25	76.1%
Grok 4.1 Fast	100	100	75	63	42	75.8%
Claude 3.5 Haiku	100	100	100	54	25	75.7%
Qwen 3 32B	94	87	80	66	48	74.8%
Mistral Large 2	100	100	100	63	0	72.5%
Claude Opus 4.5	100	89	80	66	25	72.0%
MoonshotAI: Kimi K2.5	100	100	96	58	0	70.8%
Z.AI GLM 5 Turbo	100	100	63	54	34	70.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	42	0	68.3%
Hermes 3 70B	100	100	100	42	0	68.3%
Qwen3 235B A22B Instruct 2507	100	100	100	14	10	64.7%
o4 Mini High	100	100	67	56	0	64.7%
GPT-5.4 Nano (Reasoning, Low)	100	94	88	20	16	63.6%
Mistral Medium 3.1	89	80	75	63	0	61.3%
Z.AI GLM 4.5	100	72	65	60	0	59.4%
MiniMax M2.5	100	89	66	34	0	57.9%
GPT-4.1 Mini	87	54	50	48	46	56.8%
Gemini 3 Pro (Preview)	75	69	57	48	32	56.5%
DeepSeek-V2 Chat	100	100	60	20	0	55.9%
ByteDance Seed 1.6 Flash	100	87	58	25	0	54.0%
LFM2 24B	94	92	75	0	0	52.1%
Mistral Small 4	100	100	48	0	0	49.6%
DeepSeek V3 (2024-12-26)	100	100	30	14	0	48.7%
Grok 4	80	75	48	25	0	45.5%
GPT-4o, Aug. 6th (temp=1)	96	75	42	14	0	45.2%
Claude 3 Haiku	100	66	60	0	0	45.2%
Ministral 3 14B	100	69	25	25	0	43.9%
GPT-4o, May 13th (temp=1)	100	97	18	0	0	43.0%
Z.AI GLM 4.7 Flash	95	60	48	10	0	42.4%
Inception Mercury	100	100	0	0	0	40.0%
Cohere Command R+ (Aug. 2024)	87	69	25	0	0	36.2%
Llama 3.1 Nemotron 70B	80	54	42	0	0	35.0%
GPT-4o Mini (temp=1)	72	65	36	0	0	34.6%
Stealth: Healer Alpha	100	54	11	7	0	34.5%
Mistral Large 3	94	48	30	0	0	34.3%
Llama 3.1 8B	83	54	20	0	0	31.3%
Gemini 3 Flash (Preview, Reasoning)	100	31	10	0	0	28.2%
DeepSeek V3.2	82	58	0	0	0	28.0%
Z.AI GLM 4.7	57	54	29	0	0	27.8%
Z.AI GLM 4.6	80	34	18	0	0	26.3%
Mistral Small Creative	100	25	0	0	0	25.0%
Gemini 2.5 Flash (Reasoning)	48	32	31	6	6	24.7%
Ministral 3 8B	100	14	7	0	0	24.2%
Mistral NeMO	100	18	0	0	0	23.6%
Gemini 2.5 Flash Lite	69	45	0	0	0	22.9%
Grok 4 Fast	48	42	18	0	0	21.5%
Gemini 3 Flash (Preview)	57	42	5	0	0	20.7%
Gemini 2.5 Pro	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
DeepSeek V3.1	48	34	10	0	0	18.4%
Gemini 2.5 Flash	63	28	0	0	0	18.1%
GPT-4.1 Nano	87	0	0	0	0	17.3%
Ministral 8B	87	0	0	0	0	17.3%
GPT-4o, May 13th (temp=0)	58	12	10	0	0	16.0%
Gemini 3.1 Flash Lite (Preview)	66	0	0	0	0	13.2%
Gemma 3 12B	30	18	16	0	0	12.8%
Aion 2.0	37	14	6	0	0	11.3%
Stealth: Aurora Alpha	48	0	0	0	0	9.6%
GPT-4o, Aug. 6th (temp=0)	48	0	0	0	0	9.6%
Qwen 2.5 72B	48	0	0	0	0	9.6%
Qwen 3.5 Plus (2026-02-15)	31	12	4	0	0	9.4%
Gemma 3 27B	25	6	0	0	0	6.2%
Ministral 3 3B	25	0	0	0	0	5.0%
Arcee AI: Trinity Mini	18	0	0	0	0	3.6%
Nemotron 3 Nano	11	0	0	0	0	2.3%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3B	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	96	99.1%
MiniMax M2.7	100	100	100	100	94	98.8%
GPT-5.2	100	100	100	99	91	97.8%
Qwen3 235B A22B Instruct 2507	100	100	100	97	92	97.8%
GPT-5	100	100	100	100	86	97.1%
DeepSeek V3 (2025-03-24)	100	100	100	100	83	96.7%
GPT-5.1	100	100	100	100	78	95.6%
Claude Sonnet 4.6	100	100	100	89	87	95.2%
Writer: Palmyra X5	100	100	100	83	83	93.3%
MoonshotAI: Kimi K2.5	100	100	100	89	69	91.7%
GPT-5.4 (Reasoning)	100	100	100	100	54	90.7%
Qwen 3.5 122B	100	100	100	77	75	90.5%
Z.AI GLM 5 Turbo	100	100	96	89	65	90.0%
MiniMax M2.5	100	100	100	69	69	87.8%
Claude Opus 4.5	100	99	94	72	63	85.5%
Claude Opus 4.6	100	100	100	63	54	83.2%
Claude 3.5 Sonnet	100	96	87	75	50	81.4%
WizardLM 2 8x22b	96	91	82	80	56	80.9%
Grok 4.20 (Beta)	100	100	81	63	60	80.8%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
GPT-5.4 Nano	100	91	82	77	51	79.9%
Claude Haiku 4.5	100	94	80	72	45	78.1%
Rocinante 12B	100	100	100	91	0	78.1%
Llama 3.1 8B	100	100	100	89	0	77.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	94	54	42	77.8%
Mistral Small 4 (Reasoning)	100	99	96	48	44	77.3%
GPT-5.4 Mini (Reasoning)	100	100	100	80	0	75.9%
Z.AI GLM 5	100	100	89	89	0	75.7%
GPT-5.4 Nano (Reasoning)	100	83	75	69	45	74.6%
Claude Sonnet 4	100	100	94	75	0	73.8%
Claude 3.7 Sonnet	100	96	75	57	35	72.7%
Grok 4.20 (Beta, Reasoning)	100	100	94	50	20	72.7%
GPT-5 Mini	94	84	74	69	38	71.9%
GPT-4o, Aug. 6th (temp=1)	97	97	94	63	7	71.6%
Gemini 3.1 Flash Lite (Preview)	100	100	92	63	0	70.8%
o4 Mini High	100	85	75	75	18	70.6%
Hermes 3 405B	100	100	96	56	0	70.3%
Gemini 3 Pro (Preview)	100	81	78	54	38	70.1%
Grok 4.1 Fast	100	99	69	69	0	67.5%
ByteDance Seed 1.6	100	100	100	25	0	65.0%
Stealth: Hunter Alpha	97	89	58	48	31	64.8%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	0	0	60.0%
DeepSeek V3.2	100	72	66	46	14	59.7%
Mistral Small 4	100	100	50	48	0	59.6%
GPT-5.4 Nano (Reasoning, Low)	95	95	48	38	12	57.5%
Qwen 3 32B	96	75	54	34	25	56.7%
Qwen 3.5 Plus (2026-02-15)	100	100	78	0	0	55.6%
ByteDance Seed 2.0 Mini	100	100	54	18	0	54.3%
Mistral Medium 3.1	100	100	31	30	0	52.2%
GPT-4.1	100	75	69	0	0	48.9%
Aion 2.0	99	65	38	25	7	46.8%
ByteDance Seed 1.6 Flash	87	57	51	31	0	45.2%
Z.AI GLM 4.5	91	69	48	14	0	44.4%
o4 Mini	100	100	20	0	0	43.9%
Mistral Large 2	100	63	25	25	0	42.5%
Z.AI GLM 4.7 Flash	100	66	46	0	0	42.4%
Gemini 2.5 Flash	82	56	42	17	8	40.9%
Grok 4	80	54	42	25	0	40.0%
Mistral Large 3	69	63	34	16	0	36.5%
Hermes 3 70B	100	80	0	0	0	35.9%
Ministral 3 14B	100	80	0	0	0	35.9%
LFM2 24B	83	75	18	0	0	35.2%
Mistral Small Creative	87	77	7	0	0	34.3%
DeepSeek V3 (2024-12-26)	85	42	42	0	0	33.7%
DeepSeek-V2 Chat	75	75	0	0	0	30.0%
Claude 3 Haiku	100	46	0	0	0	29.2%
Z.AI GLM 4.7	78	58	8	0	0	29.0%
Gemini 2.5 Flash (Reasoning)	68	50	5	4	0	25.5%
Nemotron 3 Nano	100	22	0	0	0	24.3%
Stealth: Healer Alpha	100	14	0	0	0	22.8%
GPT-4.1 Mini	57	54	0	0	0	22.1%
Gemma 3 27B	57	54	0	0	0	22.1%
GPT-4o, May 13th (temp=1)	54	31	25	0	0	22.0%
Mistral Large	50	31	25	0	0	21.3%
Cohere Command R+ (Aug. 2024)	66	34	0	0	0	20.1%
Llama 3.1 70B	100	0	0	0	0	20.0%
Grok 4 Fast	54	18	14	0	0	17.1%
Gemini 2.5 Pro	25	20	18	18	0	16.1%
Gemini 3 Flash (Preview)	75	0	0	0	0	15.0%
Llama 3.1 Nemotron 70B	42	18	0	0	0	11.9%
Gemma 3 12B	34	21	0	0	0	11.0%
GPT-4.1 Nano	54	0	0	0	0	10.7%
Ministral 3B	54	0	0	0	0	10.7%
DeepSeek V3.1	34	11	0	0	0	9.1%
Arcee AI: Trinity Large (Preview)	45	0	0	0	0	9.0%
GPT-5 Nano	25	18	0	0	0	8.6%
Gemma 3 4B	30	0	0	0	0	6.0%
Gemini 2.5 Flash Lite (Reasoning)	25	0	0	0	0	5.0%
Ministral 3 8B	16	0	0	0	0	3.2%
Z.AI GLM 4.6	14	0	0	0	0	2.8%
GPT-4o Mini (temp=1)	10	0	0	0	0	1.9%
Arcee AI: Trinity Mini	10	0	0	0	0	1.9%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
Ministral 8B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	94	98.9%
Claude 3.7 Sonnet	100	100	100	100	89	97.9%
Mistral Medium 3.1	100	100	100	100	75	95.0%
GPT-5 Mini	100	100	96	89	85	94.0%
Claude Opus 4.6 (Reasoning)	100	100	100	100	69	93.9%
Qwen 3 32B	100	100	100	83	81	92.8%
Hermes 3 405B	100	100	100	100	63	92.5%
Claude 3.5 Sonnet	100	100	100	100	54	90.7%
Gemini 2.5 Flash Lite	100	100	100	100	54	90.7%
GPT-5.1	100	97	94	82	75	89.5%
Z.AI GLM 5	100	100	100	100	42	88.3%
Rocinante 12B	100	100	100	100	42	88.3%
Z.AI GLM 4.7	100	100	100	80	48	85.5%
ByteDance Seed 2.0 Lite	100	100	100	100	25	85.0%
GPT-5.4 (Reasoning)	100	100	100	100	0	80.0%
Claude Sonnet 4.6	100	100	100	100	0	80.0%
Z.AI GLM 4.5	100	100	100	100	0	80.0%
GPT-5.4 Mini	100	100	100	100	0	80.0%
MoonshotAI: Kimi K2.5	100	100	100	63	34	79.3%
GPT-5.4 (Reasoning, Low)	100	100	100	54	42	79.0%
Claude Haiku 4.5	100	100	100	42	42	76.7%
MiniMax M2.7	100	100	92	54	34	75.9%
Stealth: Healer Alpha	100	100	58	54	42	70.7%
Gemini 2.5 Flash	100	100	100	54	0	70.7%
MiniMax M2.5	100	87	87	80	0	70.5%
Writer: Palmyra X5	100	100	100	25	25	70.0%
GPT-5.4 Nano (Reasoning)	100	99	66	48	37	69.9%
Mistral Small 4	100	100	80	42	25	69.2%
Llama 3.1 8B	100	100	100	37	0	67.4%
GPT-5.2	100	100	69	60	0	65.8%
Mistral Small 4 (Reasoning)	100	100	92	34	0	65.2%
Z.AI GLM 4.6	100	100	100	25	0	65.0%
Aion 2.0	100	100	63	42	14	63.6%
Grok 4.20 (Beta)	92	77	75	69	0	62.7%
ByteDance Seed 1.6 Flash	100	100	63	48	0	62.1%
WizardLM 2 8x22b	100	100	80	25	0	60.9%
Gemini 2.5 Pro	100	100	100	0	0	60.0%
ByteDance Seed 2.0 Mini	100	100	100	0	0	60.0%
DeepSeek V3 (2025-03-24)	100	100	100	0	0	60.0%
Mistral Large	100	100	100	0	0	60.0%
Llama 3.1 70B	100	100	100	0	0	60.0%
Hermes 3 70B	100	100	100	0	0	60.0%
Arcee AI: Trinity Mini	100	100	100	0	0	60.0%
DeepSeek V3.2	100	100	54	25	0	55.7%
Z.AI GLM 4.7 Flash	100	100	25	25	25	55.0%
Mistral Small 3.2 24B	100	100	63	0	0	52.5%
ByteDance Seed 1.6	100	100	34	0	0	46.8%
Stealth: Hunter Alpha	100	75	34	25	0	46.8%
GPT-5.4 Nano (Reasoning, Low)	80	69	32	25	14	44.1%
Grok 4 Fast	100	63	54	0	0	43.2%
GPT-4o, May 13th (temp=1)	83	69	63	0	0	43.1%
Gemini 2.5 Flash (Reasoning)	100	100	6	0	0	41.2%
o4 Mini High	100	100	0	0	0	40.0%
Grok 4	100	100	0	0	0	40.0%
Mistral Large 3	100	100	0	0	0	40.0%
GPT-4o, Aug. 6th (temp=1)	100	100	0	0	0	40.0%
Inception Mercury	100	100	0	0	0	40.0%
GPT-4.1 Nano	100	100	0	0	0	40.0%
Mistral NeMO	100	100	0	0	0	40.0%
Ministral 3B	100	100	0	0	0	40.0%
Mistral Small Creative	100	63	34	0	0	39.3%
GPT-4o Mini (temp=1)	100	58	20	11	0	37.9%
Grok 4.20 (Beta, Reasoning)	100	75	14	0	0	37.8%
Grok 4.1 Fast	100	69	14	0	0	36.7%
Ministral 3 14B	69	69	25	0	0	32.8%
Gemini 3 Pro (Preview)	97	48	18	0	0	32.6%
Llama 3.1 Nemotron 70B	100	54	0	0	0	30.7%
o4 Mini	54	42	34	14	0	28.6%
Arcee AI: Trinity Large (Preview)	100	42	0	0	0	28.3%
GPT-5.4 Nano	54	50	18	0	0	24.3%
GPT-5 Nano	34	34	29	18	0	23.0%
LFM2 24B	54	54	0	0	0	21.4%
DeepSeek-V2 Chat	100	0	0	0	0	20.0%
GPT-4.1 Mini	100	0	0	0	0	20.0%
Mistral Large 2	100	0	0	0	0	20.0%
Ministral 3 3B	100	0	0	0	0	20.0%
DeepSeek V3.1	69	25	0	0	0	18.9%
Claude 3 Haiku	54	25	0	0	0	15.7%
Gemini 3 Flash (Preview)	48	25	0	0	0	14.6%
Qwen 3.5 Plus (2026-02-15)	71	0	0	0	0	14.3%
Qwen 2.5 72B	69	0	0	0	0	13.9%
GPT-4.1	42	25	0	0	0	13.3%
Cohere Command R+ (Aug. 2024)	42	25	0	0	0	13.3%
Gemma 3 27B	25	25	14	0	0	12.8%
Gemini 3 Flash (Preview, Reasoning)	54	0	0	0	0	10.7%
Ministral 8B	54	0	0	0	0	10.7%
DeepSeek V3 (2024-12-26)	25	0	0	0	0	5.0%
Gemma 3 12B	25	0	0	0	0	5.0%
Nemotron 3 Nano	20	0	0	0	0	3.9%
GPT-4o, May 13th (temp=0)	14	0	0	0	0	2.8%
Gemini 3.1 Flash Lite (Preview)	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Ministral 3 8B	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.1	99	99	83	77	75	86.8%
Claude Sonnet 4.6 (Reasoning)	100	97	94	80	63	86.6%
MiniMax M2.5	100	82	80	80	72	82.6%
Mistral Small 4	100	100	97	72	37	81.3%
GPT-5.4 (Reasoning, Low)	100	100	71	67	66	81.0%
Claude Opus 4.6	100	100	82	63	57	80.2%
Qwen 3.5 397B A17B	100	100	100	100	0	80.0%
Rocinante 12B	100	100	100	100	0	80.0%
GPT-5.4 Mini (Reasoning)	100	100	94	63	42	79.6%
GPT-5.4 Mini (Reasoning, Low)	100	96	87	66	48	79.3%
GPT-5.4	100	97	82	75	42	79.1%
Claude Opus 4	100	93	85	73	36	77.3%
Mistral Small 4 (Reasoning)	100	100	71	57	30	71.6%
Qwen 3.5 9B	100	100	72	48	37	71.4%
GPT-5.4 Mini	100	100	75	42	37	70.7%
Qwen3 235B A22B Instruct 2507	100	99	89	65	0	70.6%
ByteDance Seed 2.0 Lite	100	100	89	63	0	70.4%
Qwen 3.5 35B	100	100	63	58	11	66.4%
GPT-5	94	91	74	74	0	66.3%
Z.AI GLM 5	100	100	89	34	0	64.7%
Claude Haiku 4.5	100	100	66	54	0	63.9%
GPT-5 Mini	80	72	69	63	25	61.8%
GPT-4o, Aug. 6th (temp=1)	100	75	75	54	0	60.7%
MoonshotAI: Kimi K2.5	100	96	72	34	0	60.4%
GPT-5.4 Nano (Reasoning)	92	82	75	38	10	59.2%
GPT-5.4 Nano	72	67	57	54	45	59.0%
Writer: Palmyra X5	100	63	60	42	25	57.8%
Hermes 3 70B	100	100	66	14	0	56.0%
Qwen 3.5 122B	100	100	50	25	0	55.0%
Hermes 3 405B	100	100	58	10	0	53.6%
GPT-5.2	71	68	64	44	18	53.2%
Claude Sonnet 4.5	75	58	58	37	37	53.0%
Grok 4.20 (Beta, Reasoning)	89	83	42	25	25	52.9%
Z.AI GLM 5 Turbo	100	89	42	31	0	52.4%
Ministral 8B	100	100	54	0	0	50.7%
Z.AI GLM 4.7 Flash	83	63	58	45	0	49.8%
ByteDance Seed 1.6	63	54	54	42	37	49.6%
Qwen 3.5 Flash	100	80	42	0	0	44.2%
MiniMax M2.7	100	63	58	0	0	44.2%
Claude Sonnet 4	80	66	54	18	0	43.4%
Aion 2.0	96	54	42	18	7	43.2%
DeepSeek-V2 Chat	63	63	48	34	0	41.4%
GPT-4.1	80	63	63	0	0	40.9%
GPT-5 Nano	75	60	51	16	0	40.5%
Gemini 3.1 Pro (Preview)	100	75	25	0	0	40.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	0	0	0	40.0%
ByteDance Seed 2.0 Mini	100	37	34	11	0	36.4%
Mistral Large	100	54	25	0	0	35.7%
o4 Mini High	100	63	14	0	0	35.3%
DeepSeek V3 (2025-03-24)	75	42	42	14	0	34.4%
Claude Opus 4.5	63	57	25	18	6	33.6%
LFM2 24B	63	63	37	0	0	32.4%
Claude 3.7 Sonnet	58	47	42	12	0	31.8%
Mistral Medium 3.1	83	31	25	18	0	31.5%
o4 Mini	80	75	0	0	0	30.9%
Llama 3.1 8B	100	54	0	0	0	30.7%
Qwen 3 32B	83	48	18	0	0	29.9%
GPT-4o, May 13th (temp=0)	54	54	42	0	0	29.8%
GPT-4o Mini (temp=1)	99	50	0	0	0	29.7%
GPT-5.4 Nano (Reasoning, Low)	48	39	31	25	0	28.6%
Mistral Large 3	100	25	18	0	0	28.6%
Grok 4	100	31	0	0	0	26.3%
Mistral Large 2	100	14	14	0	0	25.6%
Grok 4.20 (Beta)	58	34	34	0	0	25.3%
Claude 3.5 Sonnet	75	48	0	0	0	24.6%
Ministral 3 14B	69	54	0	0	0	24.6%
Z.AI GLM 4.5	60	34	25	0	0	23.8%
Qwen 3.5 27B	34	30	25	25	0	22.8%
Claude 3.5 Haiku	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
Llama 3.1 Nemotron 70B	100	0	0	0	0	20.0%
Ministral 3 8B	100	0	0	0	0	20.0%
Gemini 3 Flash (Preview)	58	37	0	0	0	19.0%
Gemini 3 Pro (Preview)	42	42	11	0	0	18.9%
GPT-4.1 Mini	75	18	0	0	0	18.6%
ByteDance Seed 1.6 Flash	57	36	0	0	0	18.5%
DeepSeek V3 (2024-12-26)	48	42	0	0	0	17.9%
Nemotron 3 Nano	63	0	0	0	0	12.5%
Grok 4.1 Fast	25	25	7	0	0	11.5%
Qwen 3.5 Plus (2026-02-15)	46	6	0	0	0	10.4%
Gemini 2.5 Flash (Reasoning)	34	18	0	0	0	10.4%
Stealth: Hunter Alpha	48	0	0	0	0	9.6%
Mistral Small Creative	34	10	0	0	0	8.7%
Z.AI GLM 4.7	34	0	0	0	0	6.8%
Gemini 2.5 Flash	34	0	0	0	0	6.8%
Claude 3 Haiku	30	0	0	0	0	6.0%
Grok 4 Fast	25	0	0	0	0	5.0%
DeepSeek V3.2	25	0	0	0	0	5.0%
Cohere Command R+ (Aug. 2024)	25	0	0	0	0	5.0%
Stealth: Healer Alpha	10	0	0	0	0	1.9%
GPT-4o, May 13th (temp=1)	6	0	0	0	0	1.2%
Gemini 3 Flash (Preview, Reasoning)	0	0	0	0	0	0.0%
Z.AI GLM 4.6	0	0	0	0	0	0.0%
Gemini 2.5 Pro	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
DeepSeek V3.1	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 27B	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Arcee AI: Trinity Large (Preview)	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
Ministral 3B	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Inception Mercury	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	94	98.8%
GPT-5	100	97	96	87	69	89.8%
GPT-5.4 (Reasoning, Low)	100	100	100	75	54	85.7%
DeepSeek V3 (2025-03-24)	100	100	100	100	25	85.0%
Claude Opus 4	100	100	100	69	42	82.2%
GPT-5.4	100	100	100	63	42	80.8%
Qwen 3.5 397B A17B	100	100	100	100	0	80.0%
Qwen 3.5 122B	100	100	100	100	0	80.0%
MoonshotAI: Kimi K2.5	100	100	100	100	0	80.0%
Qwen 3.5 27B	100	100	100	100	0	80.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	0	80.0%
MiniMax M2.5	100	100	100	100	0	80.0%
Gemini 2.5 Pro	100	100	100	100	0	80.0%
Qwen 3.5 Flash	100	100	100	100	0	80.0%
Stealth: Healer Alpha	100	100	100	100	0	80.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	0	80.0%
Mistral Large 3	100	100	100	100	0	80.0%
Claude Haiku 4.5	100	100	100	100	0	80.0%
DeepSeek V3.1	100	100	100	100	0	80.0%
DeepSeek V3.2	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	100	0	80.0%
Ministral 3 14B	100	100	100	100	0	80.0%
Ministral 8B	100	100	100	100	0	80.0%
GPT-5.4 (Reasoning)	100	100	100	63	34	79.3%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	54	42	79.0%
GPT-5.4 Nano (Reasoning)	100	100	63	63	25	70.0%
ByteDance Seed 1.6	100	100	100	42	0	68.3%
Grok 4 Fast	100	100	100	42	0	68.3%
Qwen 3 32B	100	100	94	42	0	67.1%
Grok 4.20 (Beta, Reasoning)	100	100	100	25	0	65.0%
GPT-5 Nano	100	100	100	25	0	65.0%
Mistral Small 4 (Reasoning)	100	100	100	25	0	65.0%
Writer: Palmyra X5	100	100	100	25	0	65.0%
o4 Mini High	100	100	100	0	0	60.0%
Z.AI GLM 4.6	100	100	100	0	0	60.0%
Stealth: Hunter Alpha	100	100	100	0	0	60.0%
Gemini 2.5 Flash	100	100	100	0	0	60.0%
Mistral Large	100	100	100	0	0	60.0%
Qwen3 235B A22B Instruct 2507	100	100	100	0	0	60.0%
Llama 3.1 Nemotron 70B	100	100	100	0	0	60.0%
GPT-4.1 Nano	100	100	100	0	0	60.0%
Ministral 3 8B	100	100	100	0	0	60.0%
Claude 3 Haiku	100	100	100	0	0	60.0%
Mistral NeMO	100	100	100	0	0	60.0%
GPT-5.1	100	89	75	34	0	59.7%
GPT-5 Mini	87	80	63	63	0	58.2%
Grok 4.20 (Beta)	100	100	42	42	0	56.7%
Mistral Small Creative	100	100	54	25	0	55.7%
Claude Sonnet 4	100	100	63	14	0	55.3%
ByteDance Seed 2.0 Lite	100	100	54	14	0	53.5%
Gemini 3 Pro (Preview)	100	100	54	14	0	53.5%
Hermes 3 70B	100	100	42	25	0	53.3%
ByteDance Seed 1.6 Flash	77	63	54	50	0	48.7%
Gemini 3 Flash (Preview)	100	100	42	0	0	48.3%
Hermes 3 405B	100	100	42	0	0	48.3%
o4 Mini	100	100	34	0	0	46.8%
ByteDance Seed 2.0 Mini	100	100	25	0	0	45.0%
Nemotron 3 Super	100	100	25	0	0	45.0%
Mistral Medium 3.1	100	100	25	0	0	45.0%
GPT-5.4 Nano	87	69	63	0	0	43.7%
Nemotron 3 Nano	75	63	42	25	0	40.8%
GPT-4.1	100	100	0	0	0	40.0%
Grok 4	100	100	0	0	0	40.0%
Gemini 2.5 Flash (Reasoning)	100	100	0	0	0	40.0%
GPT-4o, Aug. 6th (temp=0)	100	100	0	0	0	40.0%
GPT-4o Mini (temp=1)	100	100	0	0	0	40.0%
WizardLM 2 8x22b	100	100	0	0	0	40.0%
GPT-5.2	100	75	14	0	0	37.8%
GPT-5.4 Nano (Reasoning, Low)	63	63	54	0	0	35.7%
Grok 4.1 Fast	100	42	14	0	0	31.1%
Claude Sonnet 4.5	100	42	0	0	0	28.3%
GPT-4o, May 13th (temp=1)	100	34	0	0	0	26.8%
Aion 2.0	100	25	0	0	0	25.0%
DeepSeek-V2 Chat	100	25	0	0	0	25.0%
Claude 3.5 Sonnet	100	25	0	0	0	25.0%
Gemma 3 4B	100	25	0	0	0	25.0%
Claude 3.7 Sonnet	42	25	25	25	0	23.3%
Qwen 3.5 Plus (2026-02-15)	100	0	0	0	0	20.0%
Gemini 3.1 Flash Lite (Preview)	100	0	0	0	0	20.0%
Z.AI GLM 4.7 Flash	100	0	0	0	0	20.0%
DeepSeek V3 (2024-12-26)	100	0	0	0	0	20.0%
GPT-4.1 Mini	100	0	0	0	0	20.0%
Mistral Small 3.2 24B	100	0	0	0	0	20.0%
Gemma 3 12B	100	0	0	0	0	20.0%
Gemma 3 27B	100	0	0	0	0	20.0%
Qwen 2.5 72B	100	0	0	0	0	20.0%
Gemini 3 Flash (Preview, Reasoning)	54	25	0	0	0	15.7%
Z.AI GLM 4.5	34	0	0	0	0	6.8%
Z.AI GLM 4.7	25	0	0	0	0	5.0%
Cohere Command R+ (Aug. 2024)	25	0	0	0	0	5.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Arcee AI: Trinity Large (Preview)	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	99	99.7%
Claude Opus 4.6 (Reasoning)	100	100	100	100	96	99.3%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	95	98.9%
GPT-5.4	100	100	100	100	94	98.9%
DeepSeek V3 (2025-03-24)	100	100	100	97	92	97.8%
GPT-5.4 Mini (Reasoning)	100	100	100	99	87	97.2%
GPT-5.4 (Reasoning, Low)	100	100	100	100	84	96.9%
MiniMax M2.5	100	100	100	97	81	95.6%
Qwen 3.5 27B	100	100	100	100	77	95.5%
Grok 4.20 (Beta)	100	100	100	99	72	94.2%
Claude Sonnet 4.5	100	100	100	85	83	93.7%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	60	92.0%
GPT-5.4 Mini	100	100	100	83	71	91.0%
Qwen3 235B A22B Instruct 2507	100	100	94	83	77	90.9%
Claude Haiku 4.5	100	100	100	89	63	90.4%
Claude 3.7 Sonnet	100	99	97	81	66	88.7%
MiniMax M2.7	100	100	100	78	64	88.5%
GPT-5.1	99	96	89	87	68	87.8%
MoonshotAI: Kimi K2.5	100	100	100	75	54	85.7%
Claude Opus 4.5	100	100	95	69	64	85.7%
Mistral Small 4 (Reasoning)	100	100	87	87	54	85.3%
Qwen 3.5 9B	100	100	100	100	25	85.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	25	85.0%
GPT-5.4 Nano (Reasoning)	100	100	80	71	69	83.9%
Mistral Small 4	100	96	89	87	45	83.3%
Claude 3.5 Sonnet	100	100	75	69	67	82.4%
GPT-5.4 Nano (Reasoning, Low)	100	85	77	77	72	82.2%
Z.AI GLM 5	100	100	100	63	37	79.9%
Mistral Medium 3.1	100	95	91	89	0	74.9%
Writer: Palmyra X5	100	92	83	77	20	74.4%
Grok 4.20 (Beta, Reasoning)	92	77	69	67	66	74.4%
Qwen 3.5 35B	100	100	100	42	25	73.3%
GPT-5	97	85	74	55	53	72.8%
GPT-5.2	99	82	66	59	58	72.7%
GPT-5 Mini	100	95	73	69	25	72.5%
Qwen 3.5 Flash	100	100	96	34	25	70.9%
Claude Sonnet 4	100	100	65	45	44	70.9%
Qwen 3.5 122B	100	100	69	46	37	70.5%
Z.AI GLM 4.7	85	83	69	63	38	67.7%
o4 Mini	94	87	87	54	14	66.9%
Claude Opus 4	100	85	82	60	6	66.8%
Grok 4	96	92	66	63	0	63.2%
GPT-5.4 Nano	77	71	63	54	50	62.8%
Gemini 3.1 Pro (Preview)	100	100	100	11	0	62.3%
o4 Mini High	100	77	66	54	14	62.2%
GPT-4.1	96	60	54	54	38	60.1%
Claude 3.5 Haiku	100	100	100	0	0	60.0%
Llama 3.1 70B	100	100	100	0	0	60.0%
Llama 3.1 8B	100	100	89	0	0	57.9%
GPT-4o, Aug. 6th (temp=1)	97	58	54	34	30	54.6%
ByteDance Seed 1.6 Flash	100	75	75	22	0	54.3%
Qwen 3 32B	100	66	48	42	0	51.2%
Stealth: Healer Alpha	66	58	54	45	16	47.9%
Arcee AI: Trinity Mini	80	75	75	0	0	45.9%
Gemini 3 Flash (Preview)	96	87	46	0	0	45.6%
ByteDance Seed 2.0 Mini	100	83	25	20	0	45.6%
Mistral Small Creative	100	63	37	16	10	45.0%
Ministral 3 3B	100	100	25	0	0	45.0%
Aion 2.0	75	58	45	44	0	44.5%
Mistral Large 2	83	58	58	20	0	43.9%
Arcee AI: Trinity Large (Preview)	100	60	54	5	0	43.8%
Z.AI GLM 4.7 Flash	57	54	47	42	20	43.7%
ByteDance Seed 2.0 Lite	100	75	42	0	0	43.3%
ByteDance Seed 1.6	80	75	50	0	0	40.9%
Mistral Large 3	100	100	0	0	0	40.0%
DeepSeek-V2 Chat	100	100	0	0	0	40.0%
GPT-4.1 Nano	100	87	10	0	0	39.2%
Rocinante 12B	100	87	7	0	0	38.8%
Hermes 3 405B	100	92	0	0	0	38.3%
Gemini 3 Pro (Preview)	67	58	39	16	7	37.6%
Mistral Small 3.2 24B	100	83	0	0	0	36.7%
Grok 4.1 Fast	80	54	48	0	0	36.2%
Gemini 2.5 Flash Lite (Reasoning)	100	45	34	0	0	35.8%
GPT-5 Nano	71	60	42	0	0	34.7%
DeepSeek V3 (2024-12-26)	100	57	10	0	0	33.3%
Ministral 8B	69	63	20	14	0	33.1%
Ministral 3 14B	63	42	34	11	0	29.9%
Ministral 3B	100	48	0	0	0	29.6%
Gemini 3 Flash (Preview, Reasoning)	69	54	25	0	0	29.6%
Qwen 3.5 Plus (2026-02-15)	66	58	0	0	0	24.9%
Mistral Large	66	42	14	0	0	24.3%
WizardLM 2 8x22b	100	16	0	0	0	23.3%
Cohere Command R+ (Aug. 2024)	66	34	14	0	0	22.8%
Z.AI GLM 4.5	57	47	6	0	0	22.1%
Llama 3.1 Nemotron 70B	100	0	0	0	0	20.0%
DeepSeek V3.2	57	22	16	0	0	18.9%
Stealth: Hunter Alpha	54	25	14	0	0	18.5%
DeepSeek V3.1	34	32	21	5	0	18.5%
GPT-4o, May 13th (temp=1)	92	0	0	0	0	18.3%
Nemotron 3 Nano	46	39	4	0	0	17.9%
GPT-4.1 Mini	80	0	0	0	0	15.9%
Gemini 2.5 Flash (Reasoning)	33	30	16	0	0	15.8%
Grok 4 Fast	58	20	0	0	0	15.6%
Nemotron 3 Super	65	10	0	0	0	14.9%
Ministral 3 8B	58	10	0	0	0	13.6%
GPT-4o Mini (temp=1)	54	0	0	0	0	10.7%
Gemma 3 27B	11	11	0	0	0	4.5%
Hermes 3 70B	14	7	0	0	0	4.2%
Gemma 3 12B	19	0	0	0	0	3.8%
Gemini 2.5 Pro	10	6	0	0	0	3.1%
GPT-4o, Aug. 6th (temp=0)	6	0	0	0	0	1.2%
Z.AI GLM 4.6	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
LFM2 24B	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	87	97.3%
Claude Opus 4.6	100	100	100	100	77	95.5%
GPT-5.1	100	100	100	100	71	94.1%
GPT-5	100	100	100	100	69	93.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	69	93.9%
Claude Opus 4.6 (Reasoning)	100	97	97	94	80	93.5%
GPT-5.4 Mini (Reasoning)	100	100	100	100	66	93.2%
GPT-5.4 Mini	100	100	100	100	66	93.2%
Z.AI GLM 5 Turbo	100	100	100	89	75	92.9%
Claude 3.7 Sonnet	100	100	94	88	80	92.3%
Claude Opus 4.5	100	100	100	92	63	90.8%
GPT-5.4 (Reasoning)	100	100	100	100	54	90.7%
GPT-5.4	100	100	100	100	54	90.7%
Claude Sonnet 4.6 (Reasoning)	100	100	94	83	46	84.6%
MiniMax M2.5	100	99	96	65	63	84.4%
Claude Opus 4	100	96	94	89	34	82.5%
GPT-5.4 Nano (Reasoning, Low)	100	100	75	75	60	82.1%
Claude Sonnet 4.5	100	100	100	100	10	81.9%
GPT-5.4 Nano	94	87	86	85	51	80.3%
Grok 4.20 (Beta)	100	83	83	72	63	80.3%
o4 Mini	100	100	80	63	58	80.1%
Qwen 3.5 397B A17B	100	100	100	100	0	80.0%
Rocinante 12B	100	100	100	100	0	80.0%
Mistral Small 4 (Reasoning)	100	100	100	54	42	79.0%
Claude Haiku 4.5	100	92	87	63	54	78.9%
Qwen3 235B A22B Instruct 2507	100	96	87	75	25	76.4%
GPT-5 Mini	100	100	91	38	34	72.5%
Z.AI GLM 5	100	96	92	63	8	71.6%
MiniMax M2.7	100	100	94	60	0	70.7%
Qwen 3 32B	100	100	96	42	0	67.5%
GPT-4.1	100	87	69	63	7	65.2%
Claude 3.5 Sonnet	100	75	69	54	25	64.6%
Mistral Small 4	100	87	69	66	0	64.4%
GPT-5.4 Nano (Reasoning)	100	81	75	66	0	64.4%
Mistral Large 2	100	100	83	37	0	64.0%
Mistral Large	100	89	87	34	0	62.0%
GPT-5.2	81	75	69	44	39	61.7%
Claude Sonnet 4	100	89	54	37	25	60.9%
Hermes 3 405B	100	87	75	42	0	60.6%
Qwen 3.5 Flash	100	100	100	0	0	60.0%
Mistral Medium 3.1	100	96	63	25	14	59.4%
o4 Mini High	89	89	63	54	0	58.9%
Qwen 3.5 122B	100	100	60	25	0	57.0%
Mistral Large 3	100	69	54	54	0	55.3%
Grok 4.20 (Beta, Reasoning)	100	83	63	18	10	54.7%
Qwen 3.5 9B	92	66	50	30	30	53.5%
Grok 4.1 Fast	94	80	72	14	0	51.9%
DeepSeek V3 (2025-03-24)	100	100	42	14	0	51.1%
Ministral 3 8B	100	100	34	0	0	46.8%
Writer: Palmyra X5	83	42	42	42	7	43.1%
MoonshotAI: Kimi K2.5	83	58	54	18	0	42.6%
Llama 3.1 8B	100	68	45	0	0	42.6%
Qwen 3.5 35B	100	58	25	25	0	41.7%
Gemini 3.1 Pro (Preview)	100	100	0	0	0	40.0%
ByteDance Seed 2.0 Mini	80	42	34	31	0	37.3%
ByteDance Seed 2.0 Lite	100	83	0	0	0	36.7%
DeepSeek-V2 Chat	75	54	42	0	0	34.0%
Claude 3 Haiku	100	37	31	0	0	33.6%
GPT-5 Nano	68	64	36	0	0	33.6%
LFM2 24B	96	34	31	0	0	32.2%
Llama 3.1 Nemotron 70B	100	54	0	0	0	30.7%
GPT-4o, May 13th (temp=1)	80	46	25	0	0	30.1%
Gemini 2.5 Pro	87	54	0	0	0	28.0%
Gemini 3 Flash (Preview, Reasoning)	50	46	25	19	0	28.0%
Mistral Small Creative	100	37	0	0	0	27.4%
Z.AI GLM 4.7	42	39	28	17	0	25.2%
Cohere Command R+ (Aug. 2024)	100	25	0	0	0	25.0%
Gemini 3 Pro (Preview)	77	25	11	7	5	24.9%
ByteDance Seed 1.6 Flash	68	54	0	0	0	24.3%
Qwen 3.5 27B	100	20	0	0	0	23.9%
Arcee AI: Trinity Large (Preview)	100	20	0	0	0	23.9%
Inception Mercury	91	18	2	0	0	22.1%
Grok 4 Fast	48	42	18	0	0	21.5%
Grok 4	58	34	14	0	0	21.3%
DeepSeek V3 (2024-12-26)	69	25	7	0	0	20.4%
Stealth: Aurora Alpha	100	0	0	0	0	20.0%
Claude 3.5 Haiku	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
Z.AI GLM 4.5	28	25	25	16	0	18.9%
GPT-4o Mini (temp=1)	66	18	0	0	0	16.8%
ByteDance Seed 1.6	83	0	0	0	0	16.7%
Mistral NeMO	80	0	0	0	0	15.9%
Gemini 2.5 Flash (Reasoning)	54	25	0	0	0	15.7%
GPT-4.1 Mini	42	25	7	0	0	14.8%
Stealth: Healer Alpha	50	14	0	0	0	12.8%
Hermes 3 70B	63	0	0	0	0	12.5%
Aion 2.0	37	20	0	0	0	11.3%
Ministral 3 14B	54	0	0	0	0	10.7%
WizardLM 2 8x22b	42	0	0	0	0	8.3%
Ministral 3B	42	0	0	0	0	8.3%
Ministral 8B	25	14	0	0	0	7.8%
Gemini 2.5 Flash Lite (Reasoning)	34	0	0	0	0	6.8%
Arcee AI: Trinity Mini	34	0	0	0	0	6.8%
Gemini 2.5 Flash	16	10	7	0	0	6.7%
DeepSeek V3.2	31	0	0	0	0	6.3%
Stealth: Hunter Alpha	30	0	0	0	0	6.0%
Z.AI GLM 4.7 Flash	30	0	0	0	0	6.0%
Gemma 3 27B	18	7	0	0	0	5.0%
Nemotron 3 Super	25	0	0	0	0	5.0%
Qwen 2.5 72B	25	0	0	0	0	5.0%
GPT-4.1 Nano	25	0	0	0	0	5.0%
Inception Mercury 2	20	0	0	0	0	3.9%
Mistral Small 3.2 24B	20	0	0	0	0	3.9%
Gemini 3 Flash (Preview)	18	0	0	0	0	3.6%
GPT-4o, Aug. 6th (temp=1)	14	0	0	0	0	2.8%
Z.AI GLM 4.6	0	0	0	0	0	0.0%
Qwen 3.5 Plus (2026-02-15)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
DeepSeek V3.1	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen 3.5 397B A17B	100	100	100	100	96	99.1%
Qwen 3.5 35B	100	100	100	100	96	99.1%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	99	96	98.9%
GPT-5.4	100	100	100	100	92	98.3%
Claude Sonnet 4.6	100	100	100	96	82	95.4%
GPT-5.4 (Reasoning, Low)	100	100	95	89	88	94.4%
Claude Opus 4.6 (Reasoning)	100	100	100	96	72	93.6%
GPT-5.4 (Reasoning)	100	100	96	91	76	92.5%
Claude Sonnet 4.6 (Reasoning)	100	100	100	80	80	91.8%
Z.AI GLM 5	100	100	99	83	73	91.1%
Llama 3.1 8B	100	100	96	87	72	90.8%
GPT-5 Mini	100	100	100	92	54	89.0%
Claude Opus 4.6	100	100	100	97	42	87.8%
DeepSeek V3 (2025-03-24)	100	100	100	75	63	87.5%
Z.AI GLM 5 Turbo	100	100	100	80	45	84.9%
GPT-5.4 Mini (Reasoning)	100	100	82	75	67	84.8%
GPT-5.1	100	92	83	76	72	84.4%
MiniMax M2.5	100	100	100	69	37	81.2%
MiniMax M2.7	100	98	85	72	48	80.7%
Qwen3 235B A22B Instruct 2507	100	89	72	71	69	80.5%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Rocinante 12B	100	100	100	100	0	80.0%
GPT-5	100	100	100	74	25	79.8%
GPT-5.4 Mini	92	80	80	75	65	78.2%
Claude 3.7 Sonnet	89	87	83	80	45	76.7%
Claude 3.5 Sonnet	100	100	83	63	37	76.5%
Claude Sonnet 4	100	85	80	67	48	76.0%
Claude Opus 4	100	87	75	64	42	73.5%
Claude Opus 4.5	100	89	77	49	48	72.8%
Grok 4.20 (Beta, Reasoning)	100	73	66	50	42	66.2%
ByteDance Seed 2.0 Lite	100	83	63	54	25	64.9%
Claude Sonnet 4.5	100	73	67	54	30	64.8%
GPT-5.4 Nano	100	90	54	47	29	63.9%
GPT-5.4 Nano (Reasoning)	84	76	65	50	39	63.0%
MoonshotAI: Kimi K2.5	100	89	63	58	0	62.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	0	0	60.0%
Llama 3.1 Nemotron 70B	100	75	63	34	25	59.3%
Claude Haiku 4.5	80	75	72	57	11	59.0%
Mistral Small 4 (Reasoning)	89	80	54	48	22	58.4%
GPT-5.4 Nano (Reasoning, Low)	99	80	76	33	0	57.8%
Gemini 3.1 Pro (Preview)	100	100	66	0	0	53.2%
GPT-5.2	80	58	55	34	32	51.7%
Qwen 3.5 Flash	100	66	48	34	0	49.7%
Mistral Small 4	75	69	69	25	0	47.8%
Mistral Large 2	87	75	34	20	14	45.9%
Hermes 3 70B	87	66	42	34	0	45.7%
GPT-4.1	80	66	42	31	10	45.7%
Qwen 3.5 9B	100	69	57	0	0	45.3%
Grok 4 Fast	100	69	54	0	0	44.6%
Arcee AI: Trinity Large (Preview)	100	63	42	18	0	44.4%
Hermes 3 405B	69	69	58	14	7	43.7%
Qwen 3 32B	87	63	42	18	0	41.7%
Grok 4.20 (Beta)	80	77	31	18	0	41.2%
Mistral Small Creative	100	75	25	0	0	40.0%
Ministral 3 14B	100	80	20	0	0	39.9%
Qwen 3.5 27B	100	54	25	0	0	35.7%
GPT-5 Nano	56	54	42	19	5	35.1%
ByteDance Seed 1.6	100	51	14	0	0	33.0%
Qwen 3.5 122B	100	46	14	0	0	32.0%
ByteDance Seed 2.0 Mini	100	31	25	0	0	31.3%
Gemini 3 Pro (Preview)	54	34	30	16	11	28.9%
Mistral Medium 3.1	72	54	18	0	0	28.8%
Writer: Palmyra X5	66	42	18	14	0	27.9%
Mistral Large 3	54	48	34	0	0	27.1%
GPT-4o, Aug. 6th (temp=1)	69	42	20	0	0	26.2%
ByteDance Seed 1.6 Flash	63	36	28	0	0	25.2%
Stealth: Hunter Alpha	100	20	0	0	0	23.9%
Z.AI GLM 4.7	51	30	29	8	0	23.6%
DeepSeek V3 (2024-12-26)	100	18	0	0	0	23.6%
Stealth: Healer Alpha	63	29	25	0	0	23.3%
Grok 4.1 Fast	46	37	32	0	0	23.0%
o4 Mini	54	48	0	0	0	20.3%
Grok 4	100	0	0	0	0	20.0%
GPT-4o, May 13th (temp=1)	100	0	0	0	0	20.0%
LFM2 24B	63	34	0	0	0	19.3%
WizardLM 2 8x22b	89	0	0	0	0	17.9%
Gemini 2.5 Flash Lite (Reasoning)	48	30	7	0	0	17.0%
Gemma 3 27B	54	31	0	0	0	17.0%
Gemini 2.5 Flash	71	0	0	0	0	14.3%
Aion 2.0	39	29	0	0	0	13.5%
Inception Mercury	67	0	0	0	0	13.4%
Z.AI GLM 4.5	51	14	0	0	0	13.1%
Z.AI GLM 4.7 Flash	32	16	14	0	0	12.5%
Llama 3.1 70B	63	0	0	0	0	12.5%
DeepSeek V3.2	54	7	0	0	0	12.2%
Mistral Large	54	0	0	0	0	10.7%
Gemini 2.5 Pro	46	0	0	0	0	9.2%
Nemotron 3 Super	42	0	0	0	0	8.3%
GPT-4.1 Nano	25	0	0	0	0	5.0%
Ministral 3B	25	0	0	0	0	5.0%
o4 Mini High	20	0	0	0	0	3.9%
DeepSeek V3.1	20	0	0	0	0	3.9%
GPT-4.1 Mini	10	10	0	0	0	3.8%
Cohere Command R+ (Aug. 2024)	18	0	0	0	0	3.6%
Gemini 2.5 Flash Lite	14	0	0	0	0	2.8%
Gemini 3 Flash (Preview, Reasoning)	11	0	0	0	0	2.3%
Qwen 3.5 Plus (2026-02-15)	6	0	0	0	0	1.2%
Nemotron 3 Nano	6	0	0	0	0	1.2%
Z.AI GLM 4.6	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview)	0	0	0	0	0	0.0%
DeepSeek-V2 Chat	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Ministral 3 8B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
Ministral 8B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	69	93.9%
GPT-5.4 (Reasoning, Low)	100	100	100	89	75	92.9%
Claude Opus 4	100	100	94	92	75	92.1%
Llama 3.1 8B	100	100	100	100	42	88.3%
Claude Opus 4.6 (Reasoning)	100	100	100	100	0	80.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	0	80.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	0	80.0%
Claude Opus 4.6	100	100	100	100	0	80.0%
MoonshotAI: Kimi K2.5	100	100	100	100	0	80.0%
Z.AI GLM 4.6	100	100	100	100	0	80.0%
Claude Sonnet 4	100	100	100	100	0	80.0%
Grok 4 Fast	100	100	100	75	25	80.0%
Claude Haiku 4.5	100	100	100	100	0	80.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	0	80.0%
Z.AI GLM 4.5	100	100	100	66	25	78.2%
GPT-5.4 Mini	100	100	94	42	25	72.1%
GPT-5 Nano	100	100	63	48	46	71.3%
GPT-5	100	92	66	58	36	70.4%
GPT-5.4 (Reasoning)	100	100	69	54	25	69.6%
o4 Mini High	100	87	63	54	42	68.9%
GPT-5.4	100	100	58	42	42	68.3%
Claude 3.5 Sonnet	100	100	100	42	0	68.3%
GPT-5.4 Nano	100	89	89	57	0	67.1%
Writer: Palmyra X5	92	83	80	63	10	65.3%
GPT-5.1	100	100	67	42	6	63.0%
Claude Sonnet 4.5	100	100	100	14	0	62.8%
GPT-5 Mini	100	100	75	34	0	61.8%
GPT-4.1	100	100	89	14	0	60.6%
Z.AI GLM 5	100	100	100	0	0	60.0%
Qwen 3.5 Flash	100	100	100	0	0	60.0%
Stealth: Healer Alpha	100	100	100	0	0	60.0%
Hermes 3 405B	100	100	100	0	0	60.0%
Gemini 2.5 Flash Lite	100	100	100	0	0	60.0%
Llama 3.1 70B	100	100	100	0	0	60.0%
Mistral Small 4	100	100	75	25	0	60.0%
Llama 3.1 Nemotron 70B	100	100	100	0	0	60.0%
Ministral 3 14B	100	100	100	0	0	60.0%
Nemotron 3 Super	100	100	63	25	0	57.5%
Claude Opus 4.5	100	100	87	0	0	57.3%
Rocinante 12B	100	100	87	0	0	57.3%
DeepSeek V3 (2024-12-26)	100	100	69	0	0	53.9%
GPT-4o, Aug. 6th (temp=1)	100	100	42	25	0	53.3%
Qwen 3.5 9B	100	100	58	0	0	51.7%
Qwen3 235B A22B Instruct 2507	80	75	69	34	0	51.6%
Ministral 8B	100	100	54	0	0	50.7%
Mistral Small 4 (Reasoning)	100	100	25	25	0	50.0%
MiniMax M2.7	100	80	69	0	0	49.8%
GPT-5.2	100	72	63	0	0	47.0%
Grok 4.1 Fast	77	75	63	14	0	45.8%
Gemini 3.1 Flash Lite (Preview)	100	100	25	0	0	45.0%
Aion 2.0	100	100	10	0	0	41.9%
o4 Mini	63	63	54	25	0	40.7%
Grok 4.20 (Beta, Reasoning)	100	100	0	0	0	40.0%
Mistral Large 3	100	100	0	0	0	40.0%
GPT-4.1 Mini	100	100	0	0	0	40.0%
DeepSeek V3.1	100	100	0	0	0	40.0%
Mistral Large	100	100	0	0	0	40.0%
Gemma 3 27B	100	100	0	0	0	40.0%
GPT-4.1 Nano	100	100	0	0	0	40.0%
GPT-5.4 Mini (Reasoning, Low)	100	58	25	0	0	36.7%
ByteDance Seed 2.0 Mini	100	63	14	0	0	35.3%
Grok 4	100	34	25	0	0	31.8%
ByteDance Seed 1.6 Flash	96	31	31	0	0	31.8%
Qwen 3.5 27B	100	42	14	0	0	31.1%
Nemotron 3 Nano	100	25	25	0	0	30.0%
GPT-5.4 Nano (Reasoning, Low)	100	48	0	0	0	29.6%
Gemini 3 Flash (Preview, Reasoning)	100	42	0	0	0	28.3%
Mistral Small 3.2 24B	100	42	0	0	0	28.3%
Cohere Command R+ (Aug. 2024)	100	42	0	0	0	28.3%
Claude 3.7 Sonnet	77	45	18	0	0	28.0%
Stealth: Hunter Alpha	100	34	0	0	0	26.8%
Qwen 3 32B	63	51	14	0	0	25.5%
GPT-4o, May 13th (temp=1)	63	63	0	0	0	25.0%
Mistral Large 2	100	25	0	0	0	25.0%
DeepSeek V3.2	100	25	0	0	0	25.0%
GPT-5.4 Nano (Reasoning)	48	42	25	10	0	24.9%
Grok 4.20 (Beta)	48	25	25	25	0	24.6%
Mistral Medium 3.1	72	34	0	0	0	21.3%
Qwen 3.5 122B	100	0	0	0	0	20.0%
ByteDance Seed 1.6	100	0	0	0	0	20.0%
Qwen 3.5 Plus (2026-02-15)	100	0	0	0	0	20.0%
DeepSeek-V2 Chat	100	0	0	0	0	20.0%
GPT-4o, Aug. 6th (temp=0)	100	0	0	0	0	20.0%
Inception Mercury	100	0	0	0	0	20.0%
Ministral 3 8B	100	0	0	0	0	20.0%
Ministral 3 3B	100	0	0	0	0	20.0%
Ministral 3B	100	0	0	0	0	20.0%
Gemini 3 Flash (Preview)	87	10	0	0	0	19.2%
Arcee AI: Trinity Large (Preview)	54	25	0	0	0	15.7%
Z.AI GLM 4.7	63	10	0	0	0	14.4%
Z.AI GLM 4.7 Flash	66	0	0	0	0	13.2%
Hermes 3 70B	63	0	0	0	0	12.5%
LFM2 24B	54	0	0	0	0	10.7%
GPT-4o Mini (temp=1)	25	0	0	0	0	5.0%
Mistral NeMO	25	0	0	0	0	5.0%
Gemma 3 12B	14	0	0	0	0	2.8%
Mistral Small Creative	14	0	0	0	0	2.8%
Gemini 3 Pro (Preview)	7	0	0	0	0	1.5%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
ByteDance Seed 2.0 Lite	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	94	98.8%
GPT-5.4	100	100	100	100	89	97.9%
GPT-5.4 Mini	100	100	100	100	89	97.9%
Qwen 3.5 122B	100	100	100	100	83	96.7%
ByteDance Seed 1.6	100	100	100	100	80	95.9%
Qwen 3.5 Flash	100	100	100	100	77	95.5%
GPT-5.1	100	100	100	92	80	94.2%
GPT-5.4 Mini (Reasoning)	100	100	96	96	58	89.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	94	94	58	89.2%
Claude Haiku 4.5	100	100	89	87	69	89.1%
Claude Opus 4.6	100	100	92	92	58	88.3%
MoonshotAI: Kimi K2.5	100	100	100	83	48	86.3%
GPT-5.2	100	100	78	77	65	83.9%
GPT-5	100	100	87	78	50	82.9%
Claude Sonnet 4.6 (Reasoning)	100	100	100	87	25	82.3%
MiniMax M2.5	100	97	75	69	66	81.6%
ByteDance Seed 2.0 Mini	100	100	100	100	0	80.0%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Mistral Medium 3.1	100	100	87	87	25	79.6%
Claude Opus 4.6 (Reasoning)	100	100	100	50	31	76.3%
o4 Mini High	100	100	100	75	0	75.0%
Qwen 3 32B	100	96	89	80	10	74.8%
GPT-5.4 Nano (Reasoning, Low)	92	87	75	63	50	73.1%
Llama 3.1 8B	100	100	100	54	0	70.7%
o4 Mini	100	100	100	25	18	68.6%
Mistral Large 2	100	100	83	54	0	67.4%
Grok 4.1 Fast	100	100	100	34	0	66.8%
GPT-5.4 Nano (Reasoning)	100	100	77	48	5	66.1%
Z.AI GLM 5 Turbo	100	100	63	54	10	65.1%
Mistral Small 4	100	87	75	42	18	64.2%
GPT-5.4 Nano	91	87	85	28	21	62.2%
Mistral Small 4 (Reasoning)	100	80	63	42	25	61.7%
Qwen3 235B A22B Instruct 2507	83	69	66	54	34	61.3%
Writer: Palmyra X5	89	69	66	54	25	60.7%
GPT-5 Mini	100	95	56	38	12	60.1%
Mistral Large 3	100	100	100	0	0	60.0%
Claude Sonnet 4.6	100	80	63	54	0	59.1%
Gemini 3 Flash (Preview, Reasoning)	100	100	80	0	0	55.9%
Grok 4.20 (Beta)	83	77	69	30	10	53.9%
Claude Opus 4.5	99	63	63	42	0	53.1%
Gemini 3 Flash (Preview)	100	100	48	10	0	51.5%
Ministral 8B	100	100	54	0	0	50.7%
Claude Sonnet 4.5	69	66	58	48	7	49.9%
Z.AI GLM 5	100	69	54	25	0	49.6%
Claude Opus 4	96	63	54	25	0	47.3%
LFM2 24B	94	80	54	0	0	45.4%
Llama 3.1 70B	100	100	25	0	0	45.0%
Z.AI GLM 4.7 Flash	54	54	51	38	22	43.5%
Claude 3.7 Sonnet	69	57	54	25	0	41.0%
Claude Sonnet 4	87	83	25	10	0	40.9%
GPT-4.1 Nano	100	63	42	0	0	40.8%
Grok 4	100	100	0	0	0	40.0%
DeepSeek V3 (2024-12-26)	100	100	0	0	0	40.0%
Mistral Large	100	100	0	0	0	40.0%
Ministral 3 3B	100	100	0	0	0	40.0%
Aion 2.0	54	54	34	30	25	39.2%
Llama 3.1 Nemotron 70B	100	54	42	0	0	39.0%
ByteDance Seed 1.6 Flash	82	72	25	6	0	37.0%
Z.AI GLM 4.7	100	38	37	0	0	35.0%
Gemini 3 Pro (Preview)	75	54	36	6	0	34.0%
DeepSeek V3.2	100	69	0	0	0	33.9%
WizardLM 2 8x22b	92	72	0	0	0	32.8%
Ministral 3 8B	100	58	0	0	0	31.7%
Stealth: Healer Alpha	100	42	0	0	0	28.3%
Stealth: Hunter Alpha	96	42	0	0	0	27.5%
Rocinante 12B	100	37	0	0	0	27.4%
MiniMax M2.7	60	50	25	0	0	27.0%
Mistral Small Creative	75	54	0	0	0	25.7%
Arcee AI: Trinity Large (Preview)	69	58	0	0	0	25.6%
GPT-5 Nano	91	35	0	0	0	25.1%
Claude 3.5 Sonnet	83	42	0	0	0	25.0%
DeepSeek V3.1	100	18	0	0	0	23.6%
GPT-4.1	75	42	0	0	0	23.3%
Z.AI GLM 4.5	75	42	0	0	0	23.3%
Nemotron 3 Super	100	10	0	0	0	21.9%
Gemini 2.5 Pro	100	0	0	0	0	20.0%
Hermes 3 405B	100	0	0	0	0	20.0%
Gemini 2.5 Flash	100	0	0	0	0	20.0%
Ministral 3B	100	0	0	0	0	20.0%
Hermes 3 70B	92	0	0	0	0	18.3%
GPT-4o, May 13th (temp=1)	88	0	0	0	0	17.6%
Grok 4 Fast	83	0	0	0	0	16.7%
GPT-4.1 Mini	63	0	0	0	0	12.5%
Nemotron 3 Nano	63	0	0	0	0	12.5%
Ministral 3 14B	54	0	0	0	0	10.7%
Z.AI GLM 4.6	42	0	0	0	0	8.3%
GPT-4o, May 13th (temp=0)	25	0	0	0	0	5.0%
GPT-4o, Aug. 6th (temp=0)	25	0	0	0	0	5.0%
GPT-4o, Aug. 6th (temp=1)	18	0	0	0	0	3.6%
Gemma 3 12B	7	0	0	0	0	1.5%
Inception Mercury	1	0	0	0	0	0.2%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Qwen 3.5 Plus (2026-02-15)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
DeepSeek-V2 Chat	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 27B	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Cohere Command R+ (Aug. 2024)	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	99	99.7%
Stealth: Healer Alpha	100	100	100	100	69	93.9%
MiniMax M2.5	100	100	100	100	54	90.7%
GPT-5.1	100	100	100	100	48	89.6%
Ministral 3B	100	100	100	100	42	88.3%
GPT-5	100	100	96	80	54	85.7%
Claude Opus 4.6	100	100	100	100	25	85.0%
Qwen 3.5 122B	100	100	100	100	25	85.0%
GPT-5.4	100	100	100	63	54	83.2%
o4 Mini High	100	100	100	100	14	82.8%
Claude Opus 4.6 (Reasoning)	100	100	100	100	0	80.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	0	80.0%
Claude Sonnet 4.6	100	100	100	100	0	80.0%
Gemini 3 Pro (Preview)	100	100	100	100	0	80.0%
Claude Haiku 4.5	100	100	100	100	0	80.0%
Nemotron 3 Super	100	100	100	100	0	80.0%
Claude 3.5 Sonnet	100	100	100	100	0	80.0%
Grok 4.20 (Beta)	100	100	100	100	0	80.0%
Mistral Large 2	100	100	100	100	0	80.0%
Mistral Small 4 (Reasoning)	100	100	100	100	0	80.0%
DeepSeek V3.1	100	100	100	100	0	80.0%
DeepSeek V3.2	100	100	100	100	0	80.0%
Inception Mercury	100	100	100	100	0	80.0%
Mistral Medium 3.1	100	100	100	100	0	80.0%
Rocinante 12B	100	100	100	100	0	80.0%
GPT-5.4 Mini (Reasoning)	100	100	100	54	42	79.0%
Llama 3.1 8B	100	100	100	75	0	75.0%
Cohere Command R+ (Aug. 2024)	100	100	100	69	0	73.9%
GPT-5.4 Nano (Reasoning, Low)	100	94	87	63	0	68.6%
Stealth: Hunter Alpha	100	100	100	42	0	68.3%
Qwen3 235B A22B Instruct 2507	100	100	100	42	0	68.3%
Mistral Large	100	100	100	34	0	66.8%
GPT-5.4 (Reasoning, Low)	100	100	100	25	0	65.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	25	0	65.0%
Mistral Small Creative	100	100	100	25	0	65.0%
GPT-5 Mini	100	100	64	58	0	64.5%
Mistral Small 4	100	100	100	18	0	63.6%
GPT-5.4 Nano (Reasoning)	97	83	63	58	14	63.1%
GPT-5.4 Mini (Reasoning, Low)	100	100	42	42	25	61.7%
ByteDance Seed 1.6	100	100	42	42	25	61.7%
Aion 2.0	100	100	100	0	0	60.0%
Z.AI GLM 4.5	100	100	100	0	0	60.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	0	0	60.0%
Gemini 3 Flash (Preview)	100	100	100	0	0	60.0%
Z.AI GLM 4.7 Flash	100	100	100	0	0	60.0%
Hermes 3 405B	100	100	100	0	0	60.0%
Mistral Small 3.2 24B	100	100	100	0	0	60.0%
Ministral 3 8B	100	100	100	0	0	60.0%
WizardLM 2 8x22b	100	100	100	0	0	60.0%
Ministral 8B	100	100	100	0	0	60.0%
GPT-5.4 Mini	100	80	69	48	0	59.4%
Ministral 3 14B	100	100	69	0	0	53.9%
Writer: Palmyra X5	100	100	42	25	0	53.3%
GPT-5 Nano	91	89	45	42	0	53.3%
GPT-5.4 Nano	92	87	69	0	0	49.5%
Claude Sonnet 4	100	100	42	0	0	48.3%
Qwen 3 32B	100	89	42	0	0	46.2%
DeepSeek V3 (2024-12-26)	100	100	25	0	0	45.0%
Hermes 3 70B	100	100	25	0	0	45.0%
o4 Mini	100	42	25	18	18	40.5%
Z.AI GLM 4.7	100	100	0	0	0	40.0%
ByteDance Seed 2.0 Mini	100	100	0	0	0	40.0%
Qwen 3.5 Plus (2026-02-15)	100	100	0	0	0	40.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	0	0	0	40.0%
DeepSeek-V2 Chat	100	100	0	0	0	40.0%
Gemini 2.5 Flash Lite	100	100	0	0	0	40.0%
Llama 3.1 70B	100	100	0	0	0	40.0%
Nemotron 3 Nano	100	100	0	0	0	40.0%
GPT-4.1 Nano	100	100	0	0	0	40.0%
Claude 3 Haiku	100	100	0	0	0	40.0%
Ministral 3 3B	100	100	0	0	0	40.0%
GPT-5.2	100	54	34	7	0	39.0%
ByteDance Seed 1.6 Flash	100	66	25	0	0	38.2%
Llama 3.1 Nemotron 70B	100	54	25	0	0	35.7%
Grok 4.1 Fast	100	48	25	0	0	34.6%
Claude Opus 4	100	72	0	0	0	34.5%
GPT-4o, May 13th (temp=1)	100	69	0	0	0	33.9%
Claude 3.7 Sonnet	63	50	34	18	0	32.9%
Grok 4 Fast	100	34	25	0	0	31.8%
GPT-4.1 Mini	100	42	0	0	0	28.3%
GPT-4o, Aug. 6th (temp=1)	100	25	14	0	0	27.8%
GPT-4o, Aug. 6th (temp=0)	100	0	0	0	0	20.0%
Gemma 3 27B	100	0	0	0	0	20.0%
Mistral NeMO	100	0	0	0	0	20.0%
GPT-4.1	54	25	0	0	0	15.7%
GPT-4o Mini (temp=1)	58	0	0	0	0	11.7%
Grok 4	14	0	0	0	0	2.8%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Arcee AI: Trinity Large (Preview)	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	99	99.8%
Claude Sonnet 4.6	100	100	100	100	99	99.7%
Qwen 3.5 35B	100	100	100	100	97	99.4%
Claude Opus 4.6	100	100	100	100	96	99.1%
Qwen 3.5 Flash	100	100	100	100	94	98.8%
GPT-5.4 (Reasoning, Low)	100	100	100	100	92	98.3%
Claude Haiku 4.5	100	100	100	100	92	98.3%
Mistral Small 4	100	100	100	96	94	97.9%
Gemini 3.1 Pro (Preview)	100	100	100	100	89	97.9%
Claude 3.7 Sonnet	100	100	100	89	89	95.7%
Qwen 3.5 122B	100	100	100	100	77	95.5%
GPT-5.4 Nano	100	97	95	92	92	95.0%
Z.AI GLM 5	100	100	100	100	75	95.0%
Z.AI GLM 5 Turbo	100	100	97	92	83	94.4%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	69	93.9%
GPT-5 Mini	100	100	100	100	65	93.1%
Writer: Palmyra X5	100	100	100	97	66	92.7%
Claude Opus 4.6 (Reasoning)	100	100	97	89	72	91.8%
MiniMax M2.7	100	100	100	93	60	90.6%
Claude Opus 4	100	100	89	82	82	90.5%
GPT-5.1	100	100	92	82	76	90.1%
Qwen3 235B A22B Instruct 2507	100	100	100	100	50	90.0%
Claude Opus 4.5	100	100	100	87	63	89.8%
DeepSeek V3 (2025-03-24)	100	100	100	89	54	88.6%
Mistral Small 4 (Reasoning)	100	100	100	72	69	88.4%
MoonshotAI: Kimi K2.5	100	100	100	100	42	88.3%
GPT-5.4 Mini	97	97	96	77	69	87.4%
Claude Sonnet 4.5	100	100	100	66	66	86.5%
ByteDance Seed 2.0 Lite	100	100	100	100	14	82.8%
Mistral Large 2	100	100	100	87	25	82.3%
o4 Mini	100	100	94	87	25	81.1%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	96	0	79.1%
Claude 3.5 Sonnet	100	100	94	92	10	79.0%
Llama 3.1 8B	100	100	100	94	0	78.8%
Stealth: Healer Alpha	100	100	82	56	45	76.5%
LFM2 24B	100	92	89	58	42	76.2%
GPT-5.2	100	81	74	63	58	75.1%
ByteDance Seed 2.0 Mini	100	100	83	80	0	72.6%
Qwen 3 32B	100	96	80	63	25	72.5%
GPT-5.4 Mini (Reasoning)	100	100	94	66	0	72.0%
GPT-4.1	100	87	75	72	25	71.8%
o4 Mini High	100	100	100	25	14	67.8%
Z.AI GLM 4.6	100	100	63	48	25	67.1%
GPT-5.4 Mini (Reasoning, Low)	100	77	69	63	25	66.9%
GPT-5 Nano	100	100	100	31	0	66.3%
Claude Sonnet 4	100	100	75	50	0	65.0%
Hermes 3 70B	100	100	87	25	0	62.3%
Z.AI GLM 4.5	100	100	69	25	16	62.1%
WizardLM 2 8x22b	100	100	57	31	12	60.0%
Mistral Medium 3.1	99	75	58	42	25	59.7%
Grok 4.1 Fast	100	100	54	42	0	59.0%
Mistral Small Creative	100	100	89	0	0	57.9%
Gemini 3 Pro (Preview)	100	97	87	0	0	56.8%
Hermes 3 405B	100	77	63	34	0	54.8%
Grok 4.20 (Beta)	88	72	48	34	25	53.5%
Z.AI GLM 4.7 Flash	100	94	54	14	0	52.2%
DeepSeek V3.2	85	77	57	37	0	51.2%
Aion 2.0	100	99	56	0	0	50.9%
ByteDance Seed 1.6 Flash	91	89	34	21	20	50.9%
Ministral 3 14B	80	63	54	42	0	47.5%
Mistral Large 3	100	80	54	0	0	46.6%
Ministral 3 8B	100	42	37	34	20	46.5%
Gemini 3.1 Flash Lite (Preview)	100	100	31	0	0	46.3%
Arcee AI: Trinity Large (Preview)	100	89	42	0	0	46.2%
GPT-4o, May 13th (temp=1)	100	54	30	30	14	45.4%
GPT-4o, Aug. 6th (temp=1)	100	80	42	0	0	44.2%
Stealth: Hunter Alpha	100	66	54	0	0	43.9%
Gemini 3 Flash (Preview)	100	63	54	0	0	43.2%
Llama 3.1 Nemotron 70B	100	63	48	0	0	42.1%
Inception Mercury	100	100	0	0	0	40.0%
Ministral 3B	100	100	0	0	0	40.0%
Cohere Command R+ (Aug. 2024)	100	80	14	0	0	38.7%
Z.AI GLM 4.7	83	54	25	14	0	35.2%
Gemini 2.5 Pro	89	75	0	0	0	32.9%
Ministral 3 3B	100	63	0	0	0	32.5%
Qwen 3.5 Plus (2026-02-15)	77	75	5	0	0	31.5%
Gemma 3 27B	63	50	28	0	0	28.1%
DeepSeek-V2 Chat	83	31	25	0	0	27.9%
DeepSeek V3.1	63	37	20	14	0	26.6%
Gemini 2.5 Flash	66	54	0	0	0	23.9%
Gemini 2.5 Flash Lite (Reasoning)	57	50	4	0	0	22.2%
Mistral Large	100	10	0	0	0	21.9%
Grok 4	100	0	0	0	0	20.0%
Nemotron 3 Super	100	0	0	0	0	20.0%
GPT-4.1 Nano	100	0	0	0	0	20.0%
Grok 4 Fast	96	0	0	0	0	19.1%
Ministral 8B	54	42	0	0	0	19.0%
Nemotron 3 Nano	77	11	0	0	0	17.6%
Gemma 3 12B	54	0	0	0	0	10.7%
Gemini 2.5 Flash (Reasoning)	42	10	0	0	0	10.3%
Arcee AI: Trinity Mini	46	0	0	0	0	9.2%
Mistral Small 3.2 24B	42	0	0	0	0	8.3%
Rocinante 12B	34	7	0	0	0	8.3%
GPT-4o Mini (temp=1)	18	0	0	0	0	3.6%
GPT-4.1 Mini	7	7	0	0	0	2.9%
Llama 3.1 70B	14	0	0	0	0	2.8%
Gemini 2.5 Flash Lite	10	0	0	0	0	1.9%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
DeepSeek V3 (2024-12-26)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	96	99.3%
GPT-5.4 Mini	100	100	100	100	96	99.1%
GPT-5.4 Nano (Reasoning)	100	100	100	100	96	99.1%
Qwen 3.5 27B	100	100	100	100	95	99.0%
Claude Opus 4.6 (Reasoning)	100	100	100	100	94	98.8%
Grok 4.1 Fast	100	100	100	99	89	97.6%
Gemini 3.1 Pro (Preview)	100	100	100	100	83	96.7%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	83	96.7%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	75	95.0%
Mistral Large 2	100	100	100	100	69	93.9%
Claude Sonnet 4.6	100	100	100	92	75	93.3%
Claude Opus 4.6	100	100	100	100	63	92.5%
Qwen3 235B A22B Instruct 2507	100	99	92	92	69	90.3%
GPT-5 Mini	96	93	91	86	82	89.4%
o4 Mini High	100	100	96	87	63	88.9%
o4 Mini	100	100	80	80	75	86.8%
Cohere Command R+ (Aug. 2024)	100	100	100	100	25	85.0%
GPT-5.4 Nano	100	94	88	73	69	84.9%
Claude Opus 4	100	100	96	89	37	84.3%
GPT-5.4 Mini (Reasoning)	100	100	94	92	25	82.1%
DeepSeek V3 (2025-03-24)	100	100	75	75	54	80.7%
Writer: Palmyra X5	100	92	87	75	48	80.3%
Qwen 3 32B	100	100	94	58	48	80.0%
ByteDance Seed 2.0 Mini	100	100	100	100	0	80.0%
Mistral Large	100	100	100	100	0	80.0%
Grok 4.20 (Beta)	100	83	77	75	63	79.6%
Hermes 3 70B	100	100	100	46	42	77.5%
Claude Sonnet 4.5	100	100	100	80	0	75.9%
Llama 3.1 70B	100	100	100	63	0	72.5%
Rocinante 12B	100	100	100	63	0	72.5%
Claude Haiku 4.5	100	97	83	80	0	72.0%
GPT-5.1	100	100	91	66	0	71.4%
LFM2 24B	100	100	87	58	0	69.0%
Z.AI GLM 4.5	100	100	75	66	0	68.2%
Claude Opus 4.5	87	83	80	51	34	66.9%
Claude Sonnet 4	100	100	75	42	14	66.1%
Mistral Small 4	100	100	100	25	0	65.0%
ByteDance Seed 1.6 Flash	100	94	73	29	25	64.2%
GPT-4o, Aug. 6th (temp=1)	100	100	100	18	0	63.6%
Stealth: Healer Alpha	100	100	75	25	16	63.3%
Nemotron 3 Super	100	100	100	14	0	62.8%
MoonshotAI: Kimi K2.5	100	97	69	25	21	62.5%
Hermes 3 405B	100	92	75	42	0	61.7%
Claude 3.7 Sonnet	99	91	69	42	6	61.3%
Z.AI GLM 5 Turbo	100	100	100	0	0	60.0%
Mistral Large 3	100	100	100	0	0	60.0%
GPT-4.1	100	75	54	48	14	58.1%
Gemini 2.5 Pro	100	100	63	25	0	57.5%
Z.AI GLM 5	100	100	72	14	0	57.3%
WizardLM 2 8x22b	69	65	54	54	42	56.7%
Claude 3.5 Sonnet	100	100	42	34	0	55.2%
Aion 2.0	97	72	54	46	0	53.8%
MiniMax M2.5	100	63	57	42	7	53.7%
MiniMax M2.7	75	69	67	25	25	52.4%
GPT-5.2	100	92	25	25	18	51.9%
Z.AI GLM 4.7	75	69	66	34	14	51.7%
Mistral Small Creative	100	63	54	42	0	51.5%
GPT-4.1 Nano	100	63	63	31	0	51.2%
GPT-4o, May 13th (temp=1)	87	72	50	37	0	49.1%
Arcee AI: Trinity Large (Preview)	100	100	42	0	0	48.3%
Llama 3.1 8B	100	100	37	0	0	47.4%
Mistral Medium 3.1	75	69	42	42	0	45.6%
Gemini 3 Pro (Preview)	72	71	58	6	4	42.5%
Gemini 3 Flash (Preview, Reasoning)	100	75	25	0	0	40.0%
Stealth: Hunter Alpha	100	100	0	0	0	40.0%
Gemini 3.1 Flash Lite (Preview)	100	100	0	0	0	40.0%
Claude 3.5 Haiku	100	100	0	0	0	40.0%
Mistral Small 4 (Reasoning)	100	100	0	0	0	40.0%
Grok 4 Fast	100	63	34	0	0	39.3%
Claude 3 Haiku	100	50	42	0	0	38.3%
DeepSeek V3 (2024-12-26)	100	63	25	0	0	37.5%
Grok 4	100	60	7	7	0	34.9%
Ministral 3 14B	100	63	0	0	0	32.5%
Z.AI GLM 4.6	100	42	18	0	0	31.9%
Gemini 2.5 Flash (Reasoning)	94	65	0	0	0	31.8%
GPT-4.1 Mini	69	69	18	0	0	31.3%
Inception Mercury	100	34	0	0	0	26.8%
DeepSeek V3.2	63	54	10	0	0	25.1%
Llama 3.1 Nemotron 70B	75	25	14	0	0	22.8%
GPT-5 Nano	100	0	0	0	0	20.0%
Nemotron 3 Nano	100	0	0	0	0	20.0%
Ministral 3 8B	100	0	0	0	0	20.0%
Ministral 3B	100	0	0	0	0	20.0%
Ministral 8B	75	25	0	0	0	20.0%
Z.AI GLM 4.7 Flash	47	25	16	8	0	19.3%
Gemma 3 27B	66	20	0	0	0	17.2%
DeepSeek-V2 Chat	83	0	0	0	0	16.7%
Qwen 3.5 Plus (2026-02-15)	46	28	8	0	0	16.5%
Gemini 2.5 Flash Lite (Reasoning)	54	25	0	0	0	15.7%
Gemini 2.5 Flash	48	14	14	0	0	15.2%
GPT-4o Mini (temp=1)	48	20	0	0	0	13.6%
DeepSeek V3.1	42	25	0	0	0	13.3%
Inception Mercury 2	65	0	0	0	0	13.0%
GPT-4o, May 13th (temp=0)	30	25	10	0	0	12.9%
Gemma 3 12B	30	30	0	0	0	11.9%
Gemini 2.5 Flash Lite	58	0	0	0	0	11.7%
Gemini 3 Flash (Preview)	30	18	0	0	0	9.5%
Stealth: Aurora Alpha	14	0	0	0	0	2.8%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.1	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	94	98.8%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	92	98.3%
GPT-5.4	100	100	100	100	92	98.3%
Qwen 3.5 27B	100	100	100	97	94	98.2%
GPT-5.4 (Reasoning, Low)	100	100	100	100	80	95.9%
Claude Opus 4	100	100	100	97	82	95.9%
GPT-5 Mini	100	100	100	100	75	95.0%
GPT-5	100	100	100	100	72	94.5%
Claude Opus 4.6	100	100	100	92	75	93.3%
GPT-5.4 Mini (Reasoning)	100	100	100	100	66	93.2%
o4 Mini	100	100	100	87	80	93.2%
GPT-5.4 Mini	100	100	100	80	75	90.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	50	90.0%
Claude Haiku 4.5	100	100	100	100	50	90.0%
Claude Sonnet 4.6	100	100	100	87	63	89.8%
GPT-5.4 Nano (Reasoning)	100	100	100	74	73	89.4%
Z.AI GLM 5	100	100	100	99	46	88.9%
GPT-5.4 Nano	100	100	90	85	68	88.4%
GPT-5.4 (Reasoning)	100	100	100	100	42	88.3%
GPT-5.4 Nano (Reasoning, Low)	100	100	96	83	62	88.2%
DeepSeek V3 (2025-03-24)	100	100	92	80	69	88.1%
Qwen3 235B A22B Instruct 2507	100	99	94	89	54	87.1%
Claude Opus 4.6 (Reasoning)	100	92	89	89	58	85.7%
Mistral Small 4	100	100	100	89	34	84.7%
Claude Opus 4.5	100	100	82	80	51	82.4%
MiniMax M2.7	100	100	93	88	29	81.9%
MiniMax M2.5	100	100	80	67	60	81.3%
Z.AI GLM 5 Turbo	100	100	100	63	42	80.8%
Gemini 3.1 Pro (Preview)	100	100	100	100	0	80.0%
Hermes 3 70B	100	97	89	63	42	78.1%
Rocinante 12B	100	100	98	89	0	77.5%
o4 Mini High	100	100	100	54	31	77.0%
Qwen 3 32B	100	100	100	69	0	73.9%
GPT-5.2	100	98	94	75	0	73.3%
Mistral Small 4 (Reasoning)	100	88	63	63	54	73.3%
Claude 3.7 Sonnet	99	88	66	54	49	71.1%
Stealth: Hunter Alpha	100	100	89	54	7	70.0%
Mistral Small Creative	100	100	100	42	0	68.3%
Claude Sonnet 4.5	100	100	97	38	0	67.1%
ByteDance Seed 2.0 Mini	100	100	80	54	0	66.6%
MoonshotAI: Kimi K2.5	96	89	83	37	5	62.0%
Gemini 3 Pro (Preview)	100	100	85	21	0	61.2%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	0	0	60.0%
DeepSeek V3 (2024-12-26)	100	100	100	0	0	60.0%
Claude Sonnet 4	100	75	48	37	34	58.8%
Mistral Large	100	100	75	14	0	57.8%
Claude 3.5 Sonnet	99	92	63	25	0	55.6%
Writer: Palmyra X5	99	66	54	34	25	55.5%
Ministral 3 14B	100	100	25	25	25	55.0%
GPT-5 Nano	100	100	73	0	0	54.7%
DeepSeek V3.2	92	75	63	42	0	54.2%
LFM2 24B	100	63	58	42	0	52.5%
Stealth: Healer Alpha	100	89	54	16	0	51.8%
Mistral Medium 3.1	87	77	69	25	0	51.7%
GPT-4.1	92	83	75	0	0	50.0%
Z.AI GLM 4.6	100	100	42	0	0	48.3%
Grok 4.1 Fast	87	84	44	16	0	46.3%
Mistral Large 2	87	75	69	0	0	46.2%
Grok 4 Fast	100	83	25	0	0	41.7%
DeepSeek-V2 Chat	100	87	18	0	0	40.9%
ByteDance Seed 1.6 Flash	100	72	31	0	0	40.7%
Llama 3.1 8B	100	100	0	0	0	40.0%
Z.AI GLM 4.5	100	69	14	14	0	39.4%
Mistral Large 3	100	69	25	0	0	38.9%
Ministral 3B	100	87	0	0	0	37.3%
Z.AI GLM 4.7	69	58	54	0	0	36.3%
Hermes 3 405B	100	69	10	0	0	35.8%
Grok 4.20 (Beta)	66	61	25	10	7	33.8%
Llama 3.1 Nemotron 70B	63	63	25	14	0	32.8%
Gemini 2.5 Pro	87	48	25	0	0	31.9%
Gemini 2.5 Flash	75	65	16	0	0	31.3%
Nemotron 3 Super	100	54	0	0	0	30.7%
Llama 3.1 70B	100	54	0	0	0	30.7%
Aion 2.0	77	34	25	0	0	27.3%
GPT-4o Mini (temp=1)	72	42	18	0	0	26.4%
Cohere Command R+ (Aug. 2024)	94	37	0	0	0	26.1%
Arcee AI: Trinity Mini	69	42	0	0	0	22.2%
Ministral 3 3B	54	54	0	0	0	21.4%
Grok 4	80	20	7	0	0	21.3%
Qwen 3.5 Plus (2026-02-15)	63	39	0	0	0	20.3%
Claude 3.5 Haiku	100	0	0	0	0	20.0%
Mistral Small 3.2 24B	100	0	0	0	0	20.0%
Nemotron 3 Nano	100	0	0	0	0	20.0%
Ministral 8B	100	0	0	0	0	20.0%
GPT-4o, May 13th (temp=1)	89	10	0	0	0	19.8%
Z.AI GLM 4.7 Flash	58	25	14	0	0	19.4%
Gemini 2.5 Flash Lite (Reasoning)	66	14	7	0	0	17.5%
GPT-4o, Aug. 6th (temp=1)	87	0	0	0	0	17.3%
WizardLM 2 8x22b	37	31	18	0	0	17.2%
Gemini 2.5 Flash (Reasoning)	57	8	0	0	0	13.0%
Inception Mercury 2	39	18	0	0	0	11.4%
Gemini 2.5 Flash Lite	30	25	0	0	0	11.0%
Gemma 3 27B	46	0	0	0	0	9.2%
GPT-4.1 Mini	34	0	0	0	0	6.8%
Arcee AI: Trinity Large (Preview)	34	0	0	0	0	6.8%
Stealth: Aurora Alpha	25	0	0	0	0	5.0%
DeepSeek V3.1	7	0	0	0	0	1.5%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Ministral 3 8B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	92	98.3%
Qwen 3.5 27B	100	100	100	100	87	97.3%
MiniMax M2.5	100	100	100	100	87	97.3%
Z.AI GLM 4.7	100	100	100	100	83	96.7%
GPT-5 Mini	100	100	100	82	77	91.9%
Claude Opus 4	100	100	100	100	54	90.7%
GPT-5	100	100	87	82	80	89.6%
GPT-5.4 (Reasoning)	100	100	100	100	25	85.0%
Claude Sonnet 4	100	100	100	100	25	85.0%
Mistral Small 4 (Reasoning)	100	100	100	100	25	85.0%
Claude Opus 4.6 (Reasoning)	100	100	100	100	14	82.8%
GPT-5.4 Nano (Reasoning)	100	98	91	75	38	80.3%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	0	80.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	0	80.0%
Claude 3.5 Sonnet	100	100	100	100	0	80.0%
Mistral Small Creative	100	100	100	100	0	80.0%
Z.AI GLM 4.5	100	100	100	89	0	77.9%
GPT-5.4 Mini (Reasoning, Low)	100	100	80	63	42	76.7%
Writer: Palmyra X5	100	80	69	69	63	76.2%
Stealth: Hunter Alpha	100	100	100	80	0	75.9%
GPT-5.4 Nano (Reasoning, Low)	100	100	78	63	36	75.3%
GPT-4.1	100	100	100	75	0	75.0%
GPT-5.1	100	100	89	83	0	74.5%
o4 Mini	100	100	83	80	0	72.6%
Claude Sonnet 4.6 (Reasoning)	100	100	100	63	0	72.5%
o4 Mini High	100	100	100	63	0	72.5%
Claude Sonnet 4.5	100	100	100	54	0	70.7%
MiniMax M2.7	100	94	87	72	0	70.5%
DeepSeek V3.1	100	100	100	42	0	68.3%
Gemini 2.5 Flash	100	100	100	42	0	68.3%
Mistral Medium 3.1	100	100	100	42	0	68.3%
Mistral Small 4	100	100	100	42	0	68.3%
Claude 3.7 Sonnet	100	94	89	34	14	66.2%
Claude Opus 4.5	100	72	69	48	34	64.8%
Z.AI GLM 4.7 Flash	100	100	54	42	25	64.0%
GPT-5.4 Mini	100	100	48	34	25	61.4%
Z.AI GLM 4.6	100	100	100	0	0	60.0%
Claude Haiku 4.5	100	100	100	0	0	60.0%
Mistral Large 2	100	100	100	0	0	60.0%
Gemini 2.5 Flash Lite	100	100	100	0	0	60.0%
Rocinante 12B	100	100	100	0	0	60.0%
Mistral Small 3.2 24B	100	100	97	0	0	59.4%
GPT-5.4 Nano	100	83	56	29	25	58.6%
DeepSeek V3.2	100	100	69	0	0	53.9%
WizardLM 2 8x22b	100	100	69	0	0	53.9%
GPT-5.2	100	72	54	37	0	52.5%
Qwen 3 32B	100	58	54	25	25	52.4%
ByteDance Seed 2.0 Mini	100	100	54	0	0	50.7%
GPT-4o, May 13th (temp=1)	100	100	54	0	0	50.7%
Hermes 3 405B	100	100	54	0	0	50.7%
DeepSeek V3 (2025-03-24)	100	100	54	0	0	50.7%
Qwen3 235B A22B Instruct 2507	63	63	42	42	42	50.0%
Z.AI GLM 5	100	100	42	0	0	48.3%
GPT-4o, Aug. 6th (temp=1)	100	100	18	14	0	46.3%
ByteDance Seed 1.6 Flash	100	63	42	22	0	45.1%
Hermes 3 70B	100	69	54	0	0	44.6%
ByteDance Seed 1.6	100	100	0	0	0	40.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	0	0	0	40.0%
Stealth: Healer Alpha	100	100	0	0	0	40.0%
Mistral Large 3	100	100	0	0	0	40.0%
GPT-4o, May 13th (temp=0)	100	100	0	0	0	40.0%
DeepSeek-V2 Chat	100	100	0	0	0	40.0%
GPT-4o, Aug. 6th (temp=0)	100	100	0	0	0	40.0%
Mistral Large	100	100	0	0	0	40.0%
Ministral 3 14B	100	100	0	0	0	40.0%
Grok 4 Fast	100	54	42	0	0	39.0%
Grok 4.20 (Beta)	94	54	34	0	0	36.3%
LFM2 24B	100	42	25	0	0	33.3%
GPT-5 Nano	100	42	0	0	0	28.3%
Arcee AI: Trinity Large (Preview)	100	42	0	0	0	28.3%
Gemma 3 27B	69	63	0	0	0	26.4%
Llama 3.1 8B	75	42	0	0	0	23.3%
Nemotron 3 Super	100	0	0	0	0	20.0%
DeepSeek V3 (2024-12-26)	100	0	0	0	0	20.0%
Inception Mercury	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
Nemotron 3 Nano	100	0	0	0	0	20.0%
Claude 3 Haiku	100	0	0	0	0	20.0%
Cohere Command R+ (Aug. 2024)	100	0	0	0	0	20.0%
Ministral 3B	100	0	0	0	0	20.0%
Grok 4	66	10	0	0	0	15.2%
Gemma 3 12B	75	0	0	0	0	15.0%
Llama 3.1 Nemotron 70B	54	14	0	0	0	13.5%
Gemini 3 Flash (Preview)	66	0	0	0	0	13.2%
Gemini 3 Pro (Preview)	34	14	0	0	0	9.6%
Qwen 3.5 Plus (2026-02-15)	38	0	0	0	0	7.6%
GPT-4o Mini (temp=1)	31	0	0	0	0	6.3%
Ministral 8B	25	0	0	0	0	5.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Ministral 3 8B	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

Useless dialogue additions

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets