Subject-first sentence starts

Test: Bad Writing Habits

Avg. Score

35.5%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Writer: Palmyra X5	83.2%	$0.011	22.0s	50%
2	Qwen3 235B A22B Instruct 2507	81.0%	$0.0011	59.2s	48%
3	Rocinante 12B	77.8%	$0.0014	38.4s	38%
4	Llama 3.1 8B	71.0%	$0.0003	1.3m	29%
5	GPT-5.4	70.2%	$0.049	1.4m	39%
6	Mistral Small 4 (Reasoning)	60.6%	$0.0022	30.2s	25%
7	Mistral Small 4	56.1%	$0.0014	18.2s	24%
8	Ministral 3 14B	50.5%	$0.0007	11.7s	25%
9	GPT-5.4 (Reasoning, Low)	67.7%	$0.055	1.4m	36%
10	Claude Sonnet 4.5	63.6%	$0.035	38.1s	27%
11	Grok 4.1 Fast	54.6%	$0.0018	37.8s	24%
12	Mistral Small Creative	49.3%	$0.0007	9.1s	22%
13	Z.AI GLM 5	58.9%	$0.0084	1.2m	26%
14	GPT-5.4 Mini	50.0%	$0.015	16.8s	25%
15	Claude Haiku 4.5	52.5%	$0.011	21.6s	21%
16	Z.AI GLM 5 Turbo	49.6%	$0.0081	33.2s	22%
17	GPT-5.4 Mini (Reasoning, Low)	47.8%	$0.015	16.8s	23%
18	Grok 4.20	45.9%	$0.0093	45.7s	26%
19	Hermes 3 70B	57.9%	$0.0010	1.2m	20%
20	Mistral Medium 3.1	43.3%	$0.0048	36.5s	25%
21	MiniMax M2.5	52.7%	$0.0034	1.3m	23%
22	Grok 4 Fast	45.4%	$0.0017	24.1s	20%
23	Hermes 3 405B	53.5%	$0.0032	53.2s	19%
24	DeepSeek V4 Pro	51.0%	$0.0048	1.3m	23%
25	Grok 4.20 (Beta)	38.9%	$0.018	15.8s	25%
26	Claude Opus 4.7	60.1%	$0.069	30.4s	26%
27	Claude Sonnet 4	53.0%	$0.032	43.7s	22%
28	Claude Sonnet 4.6	53.3%	$0.031	39.3s	21%
29	Llama 3.1 Nemotron 70B	45.5%	$0.0038	31.7s	18%
30	Arcee AI: Trinity Large (Preview)	44.0%	$0.0000	43.6s	19%
31	Llama 3.1 70B	44.4%	$0.0015	29.4s	17%
32	Claude Opus 4.5	58.8%	$0.070	53.4s	27%
33	Claude 3 Haiku	45.6%	$0.0025	14.9s	14%
34	DeepSeek V4 Flash (Reasoning)	41.5%	$0.0007	31.1s	18%
35	Gemini 2.5 Flash Lite	34.5%	$0.0009	9.5s	18%
36	DeepSeek V4 Flash	42.5%	$0.0006	31.6s	16%
37	GPT-5.4 Mini (Reasoning)	45.0%	$0.022	28.1s	19%
38	ByteDance Seed 1.6 Flash	38.0%	$0.0013	27.3s	18%
39	Z.AI GLM 5.1	52.9%	$0.014	1.5m	20%
40	Grok 4.20 (Reasoning)	45.7%	$0.018	1.5m	25%
41	Ministral 8B	37.6%	$0.0004	10.4s	15%
42	GPT-4o, Aug. 6th (temp=1)	44.8%	$0.018	24.4s	17%
43	MiniMax M2.7	46.3%	$0.0040	1.1m	17%
44	Stealth: Healer Alpha	34.3%	$0.0000	23.7s	18%
45	Xiaomi MIMO v2.5 Pro	41.1%	$0.0085	53.5s	19%
46	DeepSeek V3 (2025-03-24)	42.0%	$0.0014	39.4s	14%
47	GPT-4.1	39.4%	$0.018	44.7s	21%
48	Ministral 3 8B	38.5%	$0.0008	19.6s	14%
49	Gemma 3 12B	36.6%	$0.0004	41.3s	17%
50	Mistral Large 2	40.1%	$0.013	29.4s	16%
51	Mistral Large 3	36.9%	$0.0033	30.3s	16%
52	Stealth: Hunter Alpha	40.0%	$0.0000	55.0s	17%
53	Gemini 2.5 Flash Lite (Reasoning)	33.3%	$0.0028	30.8s	17%
54	GPT-4o Mini (temp=1)	38.9%	$0.0012	34.8s	13%
55	Xiaomi MIMO v2.5	36.7%	$0.0054	31.8s	15%
56	Gemma 3 27B	39.8%	$0.0006	52.6s	14%
57	Claude Sonnet 4.6 (Reasoning)	53.7%	$0.060	1.2m	22%
58	Claude Opus 4.6	56.4%	$0.078	1.2m	25%
59	Qwen 3.6 Flash	38.1%	$0.010	41.4s	16%
60	GPT-4.1 Nano	34.2%	$0.0007	13.3s	12%
61	Claude Opus 4.6 (Reasoning)	58.5%	$0.088	1.4m	26%
62	Qwen 3.6 35B	37.5%	$0.0083	1.0m	17%
63	Cohere Command R+ (Aug. 2024)	45.9%	$0.020	52.5s	13%
64	Claude Opus 4.7 (Reasoning)	52.7%	$0.076	32.0s	20%
65	Mistral Large	37.3%	$0.014	30.9s	13%
66	GPT-5.4 Nano	28.8%	$0.0057	26.3s	16%
67	Gemini 2.5 Flash	28.1%	$0.0052	10.6s	13%
68	GPT-5.4 (Reasoning)	65.0%	$0.089	2.6m	29%
69	LFM2 24B	28.6%	$0.0002	28.4s	13%
70	GPT-5.4 Nano (Reasoning, Low)	26.5%	$0.0055	20.6s	15%
71	GPT-5.1	51.4%	$0.054	1.8m	22%
72	Z.AI GLM 4.6	28.9%	$0.0065	51.5s	17%
73	Gemma 3 4B	29.4%	$0.0002	20.0s	11%
74	GPT-5.4 Nano (Reasoning)	26.8%	$0.0061	24.5s	15%
75	Mistral NeMO	28.6%	$0.0005	10.1s	10%
76	Grok 4.20 (Beta, Reasoning)	34.7%	$0.039	34.0s	18%
77	Qwen 3 32B	32.1%	$0.0015	54.6s	13%
78	Gemini 2.5 Flash (Reasoning)	32.2%	$0.011	21.5s	11%
79	WizardLM 2 8x22b	38.5%	$0.0026	1.8m	16%
80	DeepSeek V4 Pro (Reasoning)	46.7%	$0.015	3.1m	23%
81	GPT-4o, May 13th (temp=1)	30.5%	$0.033	14.4s	15%
82	Gemini 3.1 Flash Lite (Reasoning)	23.5%	$0.0030	11.9s	11%
83	DeepSeek V3.2	33.7%	$0.0014	1.9m	17%
84	Ministral 3B	23.3%	$0.0001	8.1s	10%
85	o4 Mini	25.8%	$0.015	25.7s	14%
86	Z.AI GLM 4.5	29.0%	$0.0051	42.1s	12%
87	GPT-4.1 Mini	24.8%	$0.0027	19.0s	10%
88	Aion 2.0	28.8%	$0.0064	1.3m	15%
89	Z.AI GLM 4.5 Air	28.2%	$0.0029	58.2s	12%
90	Claude 3.7 Sonnet	32.1%	$0.042	46.7s	17%
91	Gemini 2.5 Pro	29.5%	$0.036	36.2s	16%
92	Qwen 3.5 Plus (2026-02-15)	22.7%	$0.0060	31.5s	12%
93	Gemini 3.1 Flash Lite (Preview)	21.3%	$0.0030	8.4s	8%
94	MoonshotAI: Kimi K2.5	43.7%	$0.019	3.2m	22%
95	GPT-5.5 (Reasoning, Low)	55.5%	$0.139	1.8m	32%
96	o4 Mini High	28.1%	$0.025	47.2s	14%
97	Grok 4	43.0%	$0.048	1.7m	16%
98	GPT-5.5	56.1%	$0.139	1.7m	29%
99	Grok 4.3	21.7%	$0.0069	30.5s	9%
100	DeepSeek V3.1	26.6%	$0.0020	1.8m	13%
101	Ministral 3 3B	18.7%	$0.0005	11.1s	5%
102	Claude 3.5 Sonnet	32.1%	$0.048	35.5s	11%
103	DeepSeek V3 (2024-12-26)	22.0%	$0.0021	54.6s	7%
104	Qwen 3.5 Plus (2026-04-20)	31.8%	$0.017	1.8m	12%
105	Gemini 3 Flash (Preview)	17.4%	$0.0078	19.6s	7%
106	DeepSeek-V2 Chat	21.7%	$0.0021	53.3s	6%
107	Gemini 3.1 Flash Lite	19.0%	$0.0030	12.1s	3%
108	GPT-4o, Aug. 6th (temp=0)	17.5%	$0.023	22.7s	8%
109	GPT-4o, May 13th (temp=0)	18.6%	$0.035	14.1s	8%
110	Qwen 3.5 397B A17B	33.8%	$0.014	3.0m	15%
111	Z.AI GLM 4.7 Flash	16.8%	$0.0017	1.2m	8%
112	Gemini 3 Flash (Preview, Reasoning)	17.3%	$0.012	30.1s	5%
113	GPT-5.5 (Reasoning)	51.9%	$0.142	1.8m	25%
114	GPT-5 Mini	17.7%	$0.0100	57.4s	7%
115	Gemma 4 26B	14.2%	$0.0009	55.1s	6%
116	Gemma 4 31B	19.2%	$0.0010	1.6m	8%
117	Qwen3.6 Max Preview	44.1%	$0.050	3.5m	19%
118	Arcee AI: Trinity Mini	11.6%	$0.0003	9.2s	0%
119	Z.AI GLM 4.7	16.6%	$0.010	1.4m	8%
120	Qwen 3.5 Flash	12.9%	$0.0025	47.5s	2%
121	Qwen 2.5 72B	8.8%	$0.0010	36.7s	2%
122	Nemotron 3 Super	13.3%	$0.0000	1.4m	5%
123	Claude Opus 4	57.5%	$0.209	1.4m	28%
124	GPT-4o Mini (temp=0)	9.5%	$0.0012	34.8s	0%
125	Gemini 3 Pro (Preview)	20.1%	$0.055	54.4s	9%
126	GPT-5.2	22.9%	$0.056	1.5m	11%
127	Stealth: Aurora Alpha	0.7%	$0.0000	9.8s	0%
128	Inception Mercury 2	1.1%	$0.0032	7.0s	0%
129	ByteDance Seed 2.0 Lite	22.0%	$0.012	2.2m	6%
130	GPT-5 Nano	13.5%	$0.0042	1.4m	2%
131	Qwen 3.5 35B	14.4%	$0.018	1.0m	1%
132	Gemma 4 31B (Reasoning)	15.2%	$0.0014	2.2m	5%
133	Qwen 3.6 27B	22.7%	$0.025	2.3m	6%
134	Inception Mercury	1.2%	$0.011	17.6s	0%
135	Nemotron 3 Nano	7.5%	$0.0010	1.1m	0%
136	Gemma 4 26B (Reasoning)	13.4%	$0.0013	2.0m	3%
137	Qwen 3.5 9B	7.6%	$0.0011	1.4m	0%
138	GPT-5	29.2%	$0.065	2.8m	13%
139	Grok 4.3 (Reasoning)	16.9%	$0.021	2.3m	5%
140	Qwen 3.5 122B	8.2%	$0.025	1.1m	0%
141	Qwen 3.5 27B	6.7%	$0.020	1.6m	0%
142	GPT-OSS 120B	0.9%	$0.0015	1.8m	0%
143	ByteDance Seed 1.6	11.8%	$0.013	2.5m	0%
144	ByteDance Seed 2.0 Mini	22.7%	$0.0045	4.9m	9%
145	Gemini 3.1 Pro (Preview)	23.9%	$0.107	1.8m	6%
146	Mistral Small 3.2 24B	25.5%	$0.0069	5.7m	9%
147	MoonshotAI: Kimi K2.6	35.9%	$0.058	6.5m	16%
35.51%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	96	88	80	92.7%
Rocinante 12B	100	100	98	83	78	91.8%
GPT-5.4 (Reasoning)	100	91	87	86	60	84.7%
GPT-5.4 (Reasoning, Low)	100	89	80	71	71	82.5%
Qwen3 235B A22B Instruct 2507	100	100	98	87	27	82.5%
Claude Sonnet 4.6 (Reasoning)	97	92	81	74	62	81.2%
Claude Sonnet 4.6	96	79	78	77	63	78.4%
Claude Sonnet 4.5	100	87	71	67	64	77.9%
Claude Opus 4.7 (Reasoning)	100	77	73	60	34	68.7%
Claude Opus 4.7	93	72	70	59	46	68.0%
Gemma 3 4B	100	78	76	42	39	67.1%
Hermes 3 70B	100	96	60	56	23	66.9%
GPT-5.4	81	75	67	60	51	66.8%
Claude Opus 4.5	83	81	72	65	29	65.9%
Claude Opus 4.6 (Reasoning)	100	72	71	39	35	63.2%
Llama 3.1 Nemotron 70B	85	73	63	41	35	59.2%
MiniMax M2.7	88	60	51	50	42	58.2%
Cohere Command R+ (Aug. 2024)	99	82	40	36	34	58.2%
Claude Opus 4	75	74	71	42	24	57.3%
Claude Opus 4.6	76	70	50	46	44	57.3%
WizardLM 2 8x22b	97	69	53	39	27	57.0%
Gemma 3 12B	87	73	53	41	31	56.9%
GPT-5.4 Mini	77	70	55	41	41	56.6%
GPT-5.5 (Reasoning, Low)	63	59	58	57	44	56.2%
Z.AI GLM 5	100	78	67	20	13	55.5%
Llama 3.1 8B	100	100	56	18	0	54.8%
GPT-5.4 Mini (Reasoning)	75	64	55	46	34	54.6%
Claude Haiku 4.5	100	61	43	36	33	54.4%
GPT-5.5	68	54	53	52	44	54.2%
Hermes 3 405B	100	100	45	14	11	54.0%
MoonshotAI: Kimi K2.5	74	60	54	48	29	53.2%
GPT-5.4 Mini (Reasoning, Low)	69	63	56	46	30	53.1%
Aion 2.0	64	59	55	43	33	50.8%
Z.AI GLM 5 Turbo	78	67	47	43	16	50.2%
Xiaomi MIMO v2.5 Pro	58	55	49	46	38	49.2%
Gemini 2.5 Pro	63	61	61	45	11	48.1%
Gemini 2.5 Flash	70	48	42	41	30	46.3%
Gemma 3 27B	59	56	55	43	18	46.2%
GPT-4o Mini (temp=1)	90	45	42	38	15	46.0%
Arcee AI: Trinity Large (Preview)	68	50	47	39	22	45.0%
Gemini 2.5 Flash Lite	52	49	43	42	37	44.7%
Xiaomi MIMO v2.5	72	53	47	44	7	44.7%
Z.AI GLM 5.1	69	67	47	41	0	44.6%
Ministral 8B	79	60	46	23	15	44.5%
DeepSeek V4 Pro (Reasoning)	73	48	43	39	18	44.2%
Gemini 2.5 Flash (Reasoning)	81	77	37	14	9	43.7%
DeepSeek V4 Pro	79	45	37	31	24	43.1%
Ministral 3 14B	75	50	43	28	18	43.1%
Claude Sonnet 4	68	54	49	29	11	42.1%
Stealth: Hunter Alpha	88	34	33	31	20	41.4%
DeepSeek V4 Flash (Reasoning)	88	51	26	22	19	41.3%
GPT-5.5 (Reasoning)	56	54	37	32	24	40.6%
Qwen 3.6 35B	73	44	38	24	23	40.4%
DeepSeek V3 (2025-03-24)	94	56	26	26	0	40.3%
GPT-5.1	54	49	38	36	25	40.2%
DeepSeek V3.2	54	52	43	39	12	40.1%
Grok 4.3	63	45	42	36	12	39.7%
Gemini 2.5 Flash Lite (Reasoning)	77	38	32	30	20	39.5%
LFM2 24B	86	61	22	21	0	37.9%
Grok 4.20 (Reasoning)	53	46	43	24	23	37.7%
Mistral Small 4 (Reasoning)	55	55	44	17	15	37.3%
DeepSeek V4 Flash	100	42	32	7	0	36.2%
GPT-4o, Aug. 6th (temp=1)	75	58	29	18	0	36.2%
Grok 4.20 (Beta)	64	45	39	21	10	35.9%
Mistral Small Creative	53	46	43	17	13	34.6%
ByteDance Seed 1.6 Flash	79	34	29	23	8	34.6%
Mistral Small 3.2 24B	81	35	29	28	0	34.6%
GPT-4.1	62	53	30	27	0	34.4%
Stealth: Healer Alpha	80	40	38	14	0	34.4%
Gemini 3 Flash (Preview, Reasoning)	48	39	35	28	16	33.2%
Ministral 3B	65	60	37	0	0	32.4%
Z.AI GLM 4.6	49	41	28	23	22	32.4%
Qwen 3.6 Flash	77	33	28	22	2	32.2%
Mistral Large 2	39	37	32	29	18	31.2%
Claude 3 Haiku	73	26	21	21	14	30.9%
Grok 4.20	54	36	31	16	16	30.6%
Gemma 4 31B (Reasoning)	35	31	31	28	27	30.4%
MoonshotAI: Kimi K2.6	58	28	28	27	10	30.0%
Llama 3.1 70B	100	31	10	7	0	29.6%
Z.AI GLM 4.5 Air	49	44	21	19	15	29.5%
Mistral Medium 3.1	46	42	31	15	13	29.2%
GPT-5.2	42	41	37	16	5	28.2%
MiniMax M2.5	73	26	25	16	0	28.0%
Grok 4 Fast	43	31	31	21	13	28.0%
Gemini 3 Pro (Preview)	49	36	27	20	7	27.6%
Claude 3.7 Sonnet	50	40	28	20	0	27.6%
Gemini 3 Flash (Preview)	38	37	36	24	0	27.0%
Mistral Large 3	46	31	20	19	17	26.4%
GPT-4o, May 13th (temp=1)	60	26	16	14	12	25.7%
Mistral Large	48	31	27	20	1	25.4%
Grok 4.1 Fast	48	27	26	17	6	24.8%
Claude 3.5 Sonnet	70	23	14	8	7	24.1%
ByteDance Seed 2.0 Lite	51	31	23	16	0	24.1%
DeepSeek V3.1	33	32	25	20	9	24.0%
Gemma 4 31B	49	43	25	0	0	23.5%
DeepSeek-V2 Chat	54	22	18	11	10	22.9%
GPT-5.4 Nano	43	28	18	14	11	22.9%
Ministral 3 8B	60	39	12	2	0	22.6%
Ministral 3 3B	45	41	13	10	0	21.8%
Qwen 3.5 397B A17B	40	31	28	8	0	21.4%
Qwen 3.6 27B	63	24	16	4	0	21.2%
GPT-5.4 Nano (Reasoning, Low)	27	27	24	20	4	20.4%
GPT-5.4 Nano (Reasoning)	37	35	14	11	5	20.3%
Qwen3.6 Max Preview	60	21	11	5	0	19.5%
Z.AI GLM 4.7	28	27	23	14	2	18.9%
ByteDance Seed 2.0 Mini	55	25	8	0	0	17.4%
o4 Mini High	46	25	9	7	0	17.4%
DeepSeek V3 (2024-12-26)	38	25	21	3	0	17.4%
Mistral Small 4	34	17	15	12	9	17.3%
GPT-5	40	18	10	9	7	16.9%
GPT-4.1 Mini	41	23	18	0	0	16.2%
Qwen 3.5 Plus (2026-04-20)	39	25	10	5	0	15.8%
Grok 4	30	28	10	6	2	15.3%
Gemini 3.1 Flash Lite (Preview)	37	17	13	5	3	15.1%
GPT-4o, Aug. 6th (temp=0)	28	23	17	4	0	14.3%
Mistral NeMO	32	27	8	0	0	13.5%
Gemma 4 26B	40	17	8	0	0	13.0%
GPT-4.1 Nano	25	20	16	2	0	12.3%
Arcee AI: Trinity Mini	43	17	0	0	0	12.0%
Qwen 3.5 9B	39	18	0	0	0	11.4%
Z.AI GLM 4.7 Flash	25	15	14	0	0	10.9%
Qwen 2.5 72B	24	19	5	3	2	10.7%
Gemini 3.1 Pro (Preview)	23	16	11	3	0	10.5%
Grok 4.20 (Beta, Reasoning)	23	13	10	2	0	9.7%
Gemma 4 26B (Reasoning)	20	15	14	0	0	9.6%
Qwen 3.5 Plus (2026-02-15)	21	14	7	3	0	9.0%
Z.AI GLM 4.5	23	13	5	4	0	9.0%
Qwen 3.5 Flash	45	0	0	0	0	9.0%
Grok 4.3 (Reasoning)	26	14	5	0	0	8.9%
GPT-4o, May 13th (temp=0)	24	16	5	0	0	8.8%
Gemini 3.1 Flash Lite	23	11	7	2	0	8.7%
GPT-5 Mini	23	8	5	0	0	7.2%
Qwen 3.5 35B	31	2	0	0	0	6.5%
GPT-5 Nano	17	9	6	0	0	6.3%
Qwen 3 32B	18	8	4	1	0	6.3%
o4 Mini	11	10	6	2	1	6.3%
GPT-4o Mini (temp=0)	27	3	1	0	0	6.1%
Nemotron 3 Super	17	13	0	0	0	6.0%
Nemotron 3 Nano	15	7	0	0	0	4.4%
Qwen 3.5 27B	19	0	0	0	0	3.8%
Gemini 3.1 Flash Lite (Reasoning)	12	6	0	0	0	3.6%
ByteDance Seed 1.6	14	0	0	0	0	2.8%
Stealth: Aurora Alpha	5	0	0	0	0	1.0%
Inception Mercury 2	4	0	0	0	0	0.9%
Qwen 3.5 122B	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	85	97.0%
MiniMax M2.7	100	100	100	94	92	97.0%
Claude Sonnet 4.5	100	100	100	99	76	95.1%
Claude Opus 4.6 (Reasoning)	100	100	100	100	69	93.9%
Claude Haiku 4.5	100	100	99	94	77	93.9%
GPT-4o Mini (temp=1)	100	100	95	95	76	93.0%
Claude Opus 4.7 (Reasoning)	100	93	93	88	82	91.2%
Claude Opus 4	100	100	100	88	57	89.1%
Rocinante 12B	100	100	100	85	60	88.9%
Z.AI GLM 5.1	100	98	95	78	70	88.2%
DeepSeek V4 Pro	100	100	100	83	57	88.0%
Claude Sonnet 4.6	99	90	89	84	76	87.5%
GPT-5.4 (Reasoning)	97	93	82	80	79	86.2%
Mistral Small 4 (Reasoning)	100	98	89	78	62	85.3%
DeepSeek V4 Flash	100	100	95	83	46	84.9%
Claude Opus 4.6	100	100	100	100	21	84.3%
Z.AI GLM 5 Turbo	97	93	89	76	66	84.2%
GPT-4o, Aug. 6th (temp=1)	100	100	95	68	57	84.1%
GPT-5.4	100	86	81	77	74	83.8%
GPT-5.4 (Reasoning, Low)	94	85	84	77	74	82.8%
Claude Opus 4.7	100	83	80	78	68	81.8%
Xiaomi MIMO v2.5 Pro	100	90	81	74	64	81.8%
Claude 3 Haiku	100	100	90	85	29	80.7%
Claude Sonnet 4	100	100	88	59	54	80.0%
Grok 4	100	96	90	65	46	79.5%
Claude Sonnet 4.6 (Reasoning)	100	100	74	59	58	78.0%
Stealth: Hunter Alpha	90	84	83	74	56	77.5%
DeepSeek V3 (2025-03-24)	100	100	90	60	37	77.4%
Llama 3.1 8B	100	100	99	77	8	76.8%
GPT-5.5	91	83	79	73	58	76.6%
Z.AI GLM 5	100	85	74	62	56	75.4%
WizardLM 2 8x22b	95	89	69	63	59	74.7%
DeepSeek V4 Flash (Reasoning)	100	100	74	58	38	73.9%
Claude Opus 4.5	100	100	82	44	42	73.7%
DeepSeek V4 Pro (Reasoning)	100	100	83	56	28	73.4%
Mistral Large 2	100	89	73	56	48	73.1%
Gemma 3 4B	99	81	81	51	50	72.4%
GPT-5.5 (Reasoning)	75	74	73	73	60	71.0%
Z.AI GLM 4.5	94	94	81	55	31	71.0%
DeepSeek V3.2	90	72	66	66	58	70.6%
Mistral Small Creative	99	85	79	52	37	70.3%
MiniMax M2.5	92	73	68	66	51	70.1%
GPT-5.4 Mini (Reasoning, Low)	85	73	67	64	59	69.8%
Arcee AI: Trinity Large (Preview)	100	91	77	48	16	66.4%
Ministral 3 14B	96	68	66	50	46	65.3%
GPT-5.1	90	71	71	52	41	65.2%
Z.AI GLM 4.5 Air	99	78	67	48	29	64.2%
Mistral Large 3	100	100	48	45	26	63.8%
Mistral Large	93	71	69	53	27	62.7%
Aion 2.0	81	67	62	60	43	62.7%
Ministral 8B	82	74	65	60	31	62.4%
GPT-5.5 (Reasoning, Low)	80	64	58	51	49	60.6%
Gemini 2.5 Flash Lite	82	74	60	41	41	59.6%
Cohere Command R+ (Aug. 2024)	100	82	72	43	0	59.5%
Mistral Medium 3.1	87	60	57	48	44	59.1%
Xiaomi MIMO v2.5	78	66	63	61	27	58.8%
Hermes 3 70B	100	92	62	38	1	58.5%
Gemma 3 12B	71	67	63	59	32	58.3%
GPT-5.4 Mini	76	60	51	48	48	56.8%
Grok 4.1 Fast	91	49	47	47	44	55.6%
GPT-4.1	86	59	51	38	37	54.1%
Gemini 2.5 Pro	67	60	60	49	34	54.0%
Gemini 3 Pro (Preview)	87	57	51	48	24	53.5%
MoonshotAI: Kimi K2.5	77	61	59	40	28	53.0%
Llama 3.1 70B	77	65	60	49	13	53.0%
GPT-5.4 Mini (Reasoning)	80	53	50	48	33	52.7%
Claude 3.7 Sonnet	67	64	54	47	32	52.6%
DeepSeek V3.1	92	80	39	31	21	52.6%
Gemini 2.5 Flash (Reasoning)	81	80	58	43	0	52.4%
Gemini 2.5 Flash	100	48	43	36	33	51.8%
DeepSeek V3 (2024-12-26)	100	72	47	19	18	51.2%
ByteDance Seed 1.6 Flash	63	55	52	51	35	51.1%
LFM2 24B	79	71	53	43	6	50.7%
GPT-4.1 Nano	72	56	50	39	35	50.4%
GPT-4o, May 13th (temp=1)	68	59	53	36	32	49.6%
Grok 4.20 (Beta)	55	48	48	46	45	48.4%
Grok 4 Fast	71	69	65	29	6	48.2%
Ministral 3 8B	75	48	48	37	34	48.2%
GPT-4.1 Mini	66	59	58	32	26	48.1%
Claude 3.5 Sonnet	100	48	47	41	1	47.4%
Mistral NeMO	100	59	39	35	3	47.2%
GPT-5.4 Nano (Reasoning, Low)	69	57	46	32	29	46.5%
Mistral Small 4	68	64	56	40	0	45.6%
GPT-5.4 Nano	66	49	47	44	19	45.2%
Gemini 2.5 Flash Lite (Reasoning)	85	80	35	24	0	44.9%
Hermes 3 405B	93	84	33	11	0	44.2%
Stealth: Healer Alpha	61	53	47	41	17	43.7%
Grok 4.20 (Reasoning)	54	50	46	39	24	42.6%
Llama 3.1 Nemotron 70B	100	53	29	17	6	41.1%
Grok 4.20	46	46	42	38	31	40.6%
DeepSeek-V2 Chat	100	64	19	13	4	39.8%
Z.AI GLM 4.6	51	48	36	32	29	39.2%
ByteDance Seed 2.0 Mini	55	43	40	29	27	38.9%
Nemotron 3 Super	73	50	46	16	9	38.8%
Z.AI GLM 4.7	74	32	32	28	28	38.7%
Qwen 3.6 27B	64	60	47	21	0	38.2%
MoonshotAI: Kimi K2.6	55	54	43	20	15	37.6%
Qwen 3 32B	52	49	45	39	1	37.2%
GPT-5	55	46	36	33	15	37.0%
GPT-5.2	57	43	36	32	17	36.9%
Arcee AI: Trinity Mini	100	41	23	6	3	34.5%
Qwen 3.5 Plus (2026-04-20)	55	48	33	22	8	33.2%
Ministral 3 3B	69	40	25	16	14	32.6%
Ministral 3B	54	38	36	24	9	32.0%
GPT-5.4 Nano (Reasoning)	51	33	32	32	9	31.4%
Qwen 3.5 Plus (2026-02-15)	58	27	25	25	13	29.7%
Grok 4.20 (Beta, Reasoning)	55	38	18	15	12	27.4%
Gemini 3.1 Pro (Preview)	42	30	25	20	19	27.4%
Gemini 3 Flash (Preview)	39	32	28	23	9	26.3%
o4 Mini	40	34	23	16	16	25.8%
GPT-4o Mini (temp=0)	37	30	28	18	15	25.4%
Z.AI GLM 4.7 Flash	33	30	22	20	17	24.4%
Gemma 4 31B	35	34	25	12	12	23.8%
o4 Mini High	39	29	25	15	11	23.7%
Qwen3.6 Max Preview	82	29	3	3	0	23.6%
Qwen 3.6 35B	46	31	22	17	0	23.2%
Grok 4.3 (Reasoning)	45	25	22	13	11	23.1%
Grok 4.3	55	38	13	7	0	22.7%
Gemma 4 31B (Reasoning)	43	30	25	8	7	22.6%
Qwen 3.6 Flash	50	34	23	4	0	22.2%
ByteDance Seed 1.6	25	23	23	15	12	19.3%
ByteDance Seed 2.0 Lite	64	12	10	10	0	19.3%
Gemma 4 26B	23	19	19	16	0	15.4%
Gemini 3 Flash (Preview, Reasoning)	35	33	7	0	0	15.1%
Gemma 4 26B (Reasoning)	46	23	5	0	0	14.8%
GPT-5 Mini	30	12	10	10	0	12.6%
GPT-4o, May 13th (temp=0)	54	5	2	0	0	12.4%
GPT-4o, Aug. 6th (temp=0)	35	16	4	2	0	11.4%
Qwen 2.5 72B	36	9	6	6	0	11.2%
GPT-5 Nano	23	17	12	2	1	11.1%
Qwen 3.5 397B A17B	16	15	8	6	6	10.1%
Gemini 3.1 Flash Lite (Reasoning)	26	4	0	0	0	6.0%
Gemini 3.1 Flash Lite (Preview)	16	6	2	0	0	4.9%
Mistral Small 3.2 24B	13	1	1	0	0	3.1%
Qwen 3.5 35B	15	0	0	0	0	3.0%
Qwen 3.5 122B	13	1	0	0	0	2.9%
Qwen 3.5 Flash	9	0	0	0	0	1.8%
Nemotron 3 Nano	9	0	0	0	0	1.8%
Gemini 3.1 Flash Lite	0	0	0	0	0	0.1%
Qwen 3.5 27B	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	78	95.7%
Qwen3 235B A22B Instruct 2507	100	100	99	82	67	89.7%
Llama 3.1 8B	100	100	100	83	40	84.5%
Claude Opus 4	100	100	100	62	50	82.5%
Rocinante 12B	100	100	100	52	2	70.8%
Mistral Small 4	95	93	68	60	13	66.1%
Mistral Small 4 (Reasoning)	79	77	68	57	9	58.1%
Claude Opus 4.7	90	70	70	29	16	55.0%
GPT-5.4	68	67	62	44	34	54.8%
Claude Opus 4.5	83	66	43	43	31	53.5%
Claude Haiku 4.5	100	53	46	35	34	53.4%
Claude Sonnet 4	100	56	47	37	25	53.0%
Mistral Large 2	85	67	47	34	28	52.3%
Ministral 3 8B	100	91	29	23	19	52.3%
Claude Sonnet 4.5	75	72	61	29	17	50.9%
Claude Sonnet 4.6 (Reasoning)	65	56	46	45	41	50.7%
DeepSeek V4 Pro	90	76	31	25	25	49.6%
Hermes 3 405B	88	66	45	39	0	47.5%
Hermes 3 70B	74	69	47	42	0	46.4%
Mistral Medium 3.1	68	61	55	45	0	45.8%
MiniMax M2.5	71	56	50	45	6	45.8%
Mistral Large 3	100	62	24	19	17	44.6%
Ministral 3 14B	85	58	43	31	5	44.2%
Llama 3.1 70B	82	48	41	24	17	42.4%
Claude Sonnet 4.6	74	58	55	12	0	40.0%
GPT-5.4 (Reasoning, Low)	59	50	44	28	17	39.8%
Claude Opus 4.6	57	48	48	24	23	39.8%
Mistral Large	75	45	42	27	4	38.6%
Z.AI GLM 5	54	48	44	27	15	37.6%
DeepSeek V4 Pro (Reasoning)	50	40	38	34	23	37.1%
Llama 3.1 Nemotron 70B	69	43	41	31	0	36.9%
Grok 4.1 Fast	71	48	48	11	0	35.8%
MoonshotAI: Kimi K2.5	54	41	28	27	26	35.0%
DeepSeek V4 Flash	81	61	27	1	0	33.9%
Qwen3.6 Max Preview	64	55	20	17	12	33.7%
GPT-5	97	54	8	3	0	32.4%
ByteDance Seed 1.6 Flash	55	31	30	23	23	32.3%
Qwen 3.5 397B A17B	50	45	34	29	0	31.6%
GPT-5.5 (Reasoning, Low)	56	31	27	26	14	31.0%
Ministral 8B	46	45	33	20	9	30.8%
Arcee AI: Trinity Large (Preview)	88	60	5	0	0	30.6%
Z.AI GLM 5 Turbo	41	37	33	22	19	30.3%
Gemma 3 12B	61	35	29	20	0	29.2%
GPT-5.5	36	34	34	25	13	28.6%
Claude Opus 4.7 (Reasoning)	55	36	34	12	0	27.4%
DeepSeek V3 (2025-03-24)	55	49	31	0	0	27.1%
Gemma 3 27B	49	37	25	12	11	27.0%
MiniMax M2.7	40	34	28	24	7	26.9%
GPT-4.1	60	41	27	0	0	25.5%
GPT-4o, Aug. 6th (temp=1)	42	39	27	17	0	25.0%
Qwen 3 32B	45	36	35	6	0	24.5%
Z.AI GLM 5.1	59	35	25	0	0	23.7%
WizardLM 2 8x22b	43	39	31	3	0	23.3%
Grok 4 Fast	53	31	18	6	5	22.8%
Grok 4.20 (Beta)	47	35	16	10	0	21.5%
GPT-5.4 Nano (Reasoning, Low)	39	24	18	15	7	20.7%
GPT-4o Mini (temp=1)	43	23	23	7	6	20.6%
GPT-5.4 (Reasoning)	39	33	31	0	0	20.6%
GPT-5.5 (Reasoning)	41	28	16	14	0	20.1%
Gemini 2.5 Flash (Reasoning)	100	0	0	0	0	20.0%
Xiaomi MIMO v2.5 Pro	50	18	17	15	0	19.9%
Grok 4.20	42	30	25	1	0	19.7%
Qwen 3.5 35B	44	40	14	0	0	19.6%
Claude 3 Haiku	71	14	13	0	0	19.6%
Z.AI GLM 4.5 Air	52	46	0	0	0	19.6%
ByteDance Seed 2.0 Lite	39	30	26	3	0	19.5%
Claude Opus 4.6 (Reasoning)	47	41	7	0	0	19.0%
Mistral Small Creative	44	37	11	2	0	19.0%
DeepSeek V3 (2024-12-26)	60	20	10	0	0	18.1%
GPT-5.1	45	25	14	3	0	17.2%
Claude 3.5 Sonnet	48	16	11	9	0	16.8%
LFM2 24B	49	16	13	5	0	16.8%
Gemma 3 4B	25	23	14	14	5	16.2%
Mistral NeMO	39	18	16	0	0	14.7%
GPT-4.1 Nano	30	22	19	2	0	14.5%
GPT-4o, May 13th (temp=1)	53	19	0	0	0	14.3%
DeepSeek-V2 Chat	38	14	11	7	0	13.8%
GPT-5.4 Nano	23	18	12	9	5	13.3%
GPT-5.4 Mini	26	20	12	7	0	13.1%
DeepSeek V4 Flash (Reasoning)	59	5	0	0	0	12.9%
GPT-5.4 Mini (Reasoning)	43	18	0	0	0	12.2%
Claude 3.7 Sonnet	34	26	0	0	0	12.0%
Grok 4.20 (Reasoning)	34	16	6	1	0	11.5%
Ministral 3B	25	18	13	0	0	11.1%
Qwen 3.6 35B	51	4	0	0	0	11.0%
MoonshotAI: Kimi K2.6	32	21	0	0	0	10.6%
Gemini 2.5 Flash Lite	31	11	5	4	0	10.3%
GPT-4o, May 13th (temp=0)	31	16	3	0	0	10.2%
GPT-5 Nano	51	0	0	0	0	10.2%
Cohere Command R+ (Aug. 2024)	41	10	0	0	0	10.2%
Qwen 3.6 Flash	38	12	0	0	0	10.1%
GPT-5.4 Nano (Reasoning)	27	13	10	0	0	10.0%
GPT-4.1 Mini	28	14	5	0	0	9.6%
Mistral Small 3.2 24B	41	5	0	0	0	9.2%
GPT-5.4 Mini (Reasoning, Low)	23	16	6	0	0	9.1%
DeepSeek V3.2	23	10	6	0	0	7.9%
Z.AI GLM 4.6	23	9	6	0	0	7.7%
Xiaomi MIMO v2.5	27	9	1	0	0	7.3%
Z.AI GLM 4.5	36	0	0	0	0	7.3%
Ministral 3 3B	34	1	0	0	0	7.1%
ByteDance Seed 1.6	31	3	0	0	0	7.0%
Aion 2.0	19	13	1	0	0	6.6%
Qwen 3.5 27B	19	14	0	0	0	6.4%
Qwen 3.5 Plus (2026-02-15)	17	15	0	0	0	6.4%
Gemini 2.5 Flash	31	0	0	0	0	6.3%
Nemotron 3 Super	31	0	0	0	0	6.1%
o4 Mini	15	11	5	0	0	6.1%
Stealth: Healer Alpha	15	9	4	1	0	5.8%
Grok 4	26	0	0	0	0	5.1%
o4 Mini High	12	9	4	0	0	5.0%
Qwen 3.5 Flash	22	3	0	0	0	4.9%
Gemini 3 Pro (Preview)	13	11	0	0	0	4.7%
GPT-5.2	23	0	0	0	0	4.5%
Qwen 3.5 Plus (2026-04-20)	15	3	0	0	0	3.6%
Qwen 2.5 72B	17	0	0	0	0	3.4%
Arcee AI: Trinity Mini	11	6	0	0	0	3.4%
GPT-4o, Aug. 6th (temp=0)	16	0	0	0	0	3.1%
Gemini 2.5 Flash Lite (Reasoning)	7	5	3	0	0	2.9%
Gemini 3.1 Pro (Preview)	13	0	0	0	0	2.6%
GPT-4o Mini (temp=0)	9	2	0	0	0	2.2%
Gemini 2.5 Pro	10	0	0	0	0	2.1%
Z.AI GLM 4.7 Flash	10	0	0	0	0	2.1%
Grok 4.20 (Beta, Reasoning)	7	1	0	0	0	1.7%
GPT-5 Mini	7	1	1	0	0	1.6%
Gemini 3.1 Flash Lite	8	0	0	0	0	1.5%
Qwen 3.6 27B	7	0	0	0	0	1.5%
Gemini 3 Flash (Preview, Reasoning)	7	0	0	0	0	1.4%
Gemma 4 26B (Reasoning)	3	1	0	0	0	0.7%
Grok 4.3	3	0	0	0	0	0.6%
Gemini 3.1 Flash Lite (Preview)	2	0	0	0	0	0.5%
ByteDance Seed 2.0 Mini	1	1	0	0	0	0.3%
Gemma 4 31B (Reasoning)	1	0	0	0	0	0.2%
Qwen 3.5 9B	1	0	0	0	0	0.2%
Stealth: Hunter Alpha	1	0	0	0	0	0.2%
Nemotron 3 Nano	0	0	0	0	0	0.1%
Grok 4.3 (Reasoning)	0	0	0	0	0	0.0%
Qwen 3.5 122B	0	0	0	0	0	0.0%
Z.AI GLM 4.7	0	0	0	0	0	0.0%
Gemma 4 31B	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Gemini 3.1 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Gemma 4 26B	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
DeepSeek V3.1	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	83	96.7%
GPT-5.4 (Reasoning, Low)	100	100	100	92	73	93.0%
Claude Opus 4.6 (Reasoning)	100	100	90	86	46	84.6%
Claude Sonnet 4.5	100	100	94	80	45	83.7%
Xiaomi MIMO v2.5 Pro	100	100	94	60	41	79.0%
Stealth: Hunter Alpha	100	93	79	66	45	76.8%
Claude Sonnet 4.6	100	98	77	60	46	76.2%
GPT-5.1	100	96	68	67	50	76.1%
Hermes 3 405B	100	96	72	58	54	76.0%
Claude Sonnet 4	100	77	72	62	57	73.7%
Mistral Small 4 (Reasoning)	100	95	65	57	52	73.7%
GPT-5.4	100	94	65	63	45	73.4%
Rocinante 12B	100	100	88	46	31	73.0%
Grok 4.1 Fast	95	84	74	53	51	71.4%
Claude Sonnet 4.6 (Reasoning)	95	90	74	48	48	71.1%
Claude Opus 4.6	100	100	59	51	45	71.0%
Claude Opus 4.7 (Reasoning)	96	89	87	44	36	70.5%
DeepSeek V4 Flash	100	73	72	65	42	70.5%
Z.AI GLM 5	100	72	65	64	52	70.5%
GPT-5.4 (Reasoning)	97	72	71	63	44	69.4%
Claude Opus 4	93	82	73	61	35	68.8%
DeepSeek V4 Flash (Reasoning)	100	87	58	53	45	68.6%
Claude Opus 4.7	100	87	76	63	13	67.9%
MoonshotAI: Kimi K2.5	100	70	65	63	39	67.4%
Mistral Small Creative	100	71	67	57	34	66.0%
GPT-5.4 Mini	82	73	62	62	51	66.0%
Llama 3.1 8B	100	100	60	50	16	65.0%
Mistral Small 4	100	66	57	57	41	64.2%
MiniMax M2.7	81	79	79	45	33	63.4%
GPT-5.5	91	73	55	55	40	62.8%
Mistral Large	92	75	66	63	7	60.7%
Z.AI GLM 5 Turbo	79	66	63	60	30	59.8%
Ministral 3 14B	95	77	63	54	9	59.6%
Hermes 3 70B	100	100	65	30	0	59.0%
GPT-5.5 (Reasoning, Low)	67	66	64	54	44	58.9%
Claude Opus 4.5	90	90	45	39	29	58.5%
Gemma 3 27B	100	71	57	44	20	58.4%
MiniMax M2.5	81	81	67	31	24	57.1%
Xiaomi MIMO v2.5	100	92	38	30	24	56.6%
Qwen3.6 Max Preview	77	60	55	45	37	54.8%
GPT-5.4 Mini (Reasoning)	59	59	59	51	45	54.6%
Arcee AI: Trinity Large (Preview)	87	62	56	42	24	54.2%
DeepSeek V4 Pro	80	65	48	43	30	53.5%
Z.AI GLM 5.1	66	63	51	50	33	52.6%
GPT-5.4 Mini (Reasoning, Low)	74	62	52	41	29	51.5%
Stealth: Healer Alpha	67	66	65	36	23	51.4%
DeepSeek V4 Pro (Reasoning)	81	57	48	46	20	50.4%
Qwen 3 32B	94	60	49	39	0	48.3%
Qwen 3.6 Flash	83	78	59	21	0	48.1%
DeepSeek V3.1	85	47	39	36	32	47.9%
Mistral Medium 3.1	72	69	59	25	13	47.6%
GPT-5	84	69	35	33	12	46.4%
GPT-5.5 (Reasoning)	65	52	51	50	12	46.3%
Claude 3.7 Sonnet	81	41	40	38	31	46.2%
WizardLM 2 8x22b	86	78	40	25	0	46.0%
Claude 3.5 Sonnet	100	57	45	19	7	45.7%
Gemma 3 4B	60	53	44	43	25	45.0%
Mistral Large 2	79	62	53	30	0	44.8%
Claude Haiku 4.5	64	59	51	48	1	44.5%
Qwen 3.6 35B	63	52	51	37	17	44.2%
Grok 4	66	59	36	31	28	43.9%
GPT-4.1 Nano	67	57	49	43	0	43.1%
DeepSeek V3 (2025-03-24)	100	43	36	21	13	42.5%
Gemma 3 12B	88	50	48	28	0	42.5%
ByteDance Seed 1.6 Flash	100	48	23	21	20	42.3%
Grok 4.20 (Beta)	58	52	34	33	33	42.0%
Gemini 2.5 Flash Lite	78	51	43	19	17	41.6%
Qwen 3.5 397B A17B	73	67	57	11	0	41.6%
Gemini 2.5 Flash (Reasoning)	68	66	47	24	3	41.5%
Grok 4.20	87	71	31	8	7	40.8%
Aion 2.0	66	50	43	25	15	40.0%
GPT-5.4 Nano	59	41	38	32	23	38.7%
Gemini 2.5 Flash Lite (Reasoning)	67	41	37	30	17	38.4%
GPT-4o, Aug. 6th (temp=1)	54	42	38	38	14	37.1%
Mistral Small 3.2 24B	84	55	44	0	0	36.6%
Mistral Large 3	84	34	28	20	16	36.5%
Qwen 3.5 Plus (2026-04-20)	65	38	28	23	18	34.2%
Gemma 4 31B	72	47	23	16	11	33.6%
Gemini 2.5 Flash	45	45	37	34	0	32.1%
GPT-5.4 Nano (Reasoning, Low)	58	53	18	16	13	31.8%
Ministral 3 8B	59	32	28	25	11	31.1%
GPT-4o Mini (temp=1)	57	50	37	12	0	31.1%
MoonshotAI: Kimi K2.6	55	31	30	23	15	30.9%
Ministral 8B	87	42	14	5	0	29.8%
Grok 4.20 (Reasoning)	60	44	23	11	4	28.6%
LFM2 24B	62	30	24	14	12	28.3%
GPT-4o, May 13th (temp=1)	69	34	22	15	0	27.9%
Gemini 3 Pro (Preview)	59	40	25	12	1	27.3%
o4 Mini High	79	22	18	8	0	25.6%
DeepSeek V3.2	40	38	34	15	1	25.6%
Cohere Command R+ (Aug. 2024)	63	53	4	4	0	24.9%
Mistral NeMO	42	30	22	19	11	24.9%
GPT-4.1	41	39	38	0	0	23.6%
Grok 4 Fast	48	33	23	11	1	23.2%
Claude 3 Haiku	77	11	10	9	3	22.0%
Gemini 3.1 Flash Lite (Preview)	37	26	22	16	9	21.9%
Grok 4.3 (Reasoning)	50	20	20	19	0	21.9%
Arcee AI: Trinity Mini	75	23	9	0	0	21.2%
Grok 4.3	42	32	16	14	1	20.9%
Z.AI GLM 4.7	61	28	12	2	0	20.7%
Z.AI GLM 4.6	52	32	11	5	2	20.5%
Ministral 3B	52	26	16	9	0	20.4%
Gemini 3 Flash (Preview)	37	31	18	13	0	19.9%
Ministral 3 3B	38	24	16	12	8	19.5%
Gemini 2.5 Pro	51	23	11	11	2	19.4%
GPT-4o, May 13th (temp=0)	44	29	19	4	0	19.1%
o4 Mini	35	30	21	7	0	18.6%
GPT-5.4 Nano (Reasoning)	44	38	4	1	0	17.5%
Gemini 3 Flash (Preview, Reasoning)	38	23	18	8	0	17.4%
Llama 3.1 70B	31	29	25	0	0	17.1%
Z.AI GLM 4.7 Flash	39	32	10	3	0	17.0%
Z.AI GLM 4.5	47	25	9	2	0	16.5%
GPT-5.2	28	26	19	9	0	16.3%
DeepSeek V3 (2024-12-26)	40	25	10	0	0	15.0%
Qwen 3.5 Plus (2026-02-15)	27	26	20	1	0	14.8%
Grok 4.20 (Beta, Reasoning)	34	21	10	8	0	14.7%
Qwen 3.6 27B	65	5	0	0	0	13.9%
Z.AI GLM 4.5 Air	27	26	14	0	0	13.4%
GPT-4o, Aug. 6th (temp=0)	35	21	10	0	0	13.3%
Gemma 4 26B	25	14	12	11	0	12.2%
Qwen 2.5 72B	47	8	4	0	0	11.9%
Gemini 3.1 Pro (Preview)	23	17	15	3	1	11.7%
Gemini 3.1 Flash Lite (Reasoning)	27	20	5	4	0	11.3%
Llama 3.1 Nemotron 70B	46	5	0	0	0	10.3%
GPT-5 Mini	22	12	7	0	0	8.1%
ByteDance Seed 1.6	20	18	0	0	0	7.5%
Qwen 3.5 35B	28	0	0	0	0	5.5%
Gemini 3.1 Flash Lite	18	5	2	0	0	5.0%
Gemma 4 31B (Reasoning)	13	10	0	0	0	4.6%
Nemotron 3 Super	22	0	0	0	0	4.4%
DeepSeek-V2 Chat	9	8	4	0	0	4.1%
Qwen 3.5 122B	16	0	0	0	0	3.3%
Gemma 4 26B (Reasoning)	11	5	0	0	0	3.2%
ByteDance Seed 2.0 Mini	12	0	0	0	0	2.4%
Qwen 3.5 27B	10	0	0	0	0	2.0%
GPT-4.1 Mini	9	0	0	0	0	1.7%
GPT-4o Mini (temp=0)	3	0	0	0	0	0.6%
GPT-OSS 120B	3	0	0	0	0	0.6%
ByteDance Seed 2.0 Lite	3	0	0	0	0	0.6%
Qwen 3.5 Flash	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Rocinante 12B	100	100	100	78	61	87.8%
Writer: Palmyra X5	100	96	82	76	43	79.4%
Llama 3.1 8B	100	100	66	60	57	76.7%
Hermes 3 70B	100	93	93	77	20	76.6%
Cohere Command R+ (Aug. 2024)	100	100	80	74	0	70.8%
Claude Opus 4.7	100	83	81	76	11	70.1%
Claude 3 Haiku	83	81	71	67	38	68.1%
Mistral Medium 3.1	86	84	74	51	37	66.4%
Claude Opus 4	98	78	54	47	43	64.0%
GPT-5.4 (Reasoning, Low)	74	68	61	58	53	62.7%
Claude Opus 4.5	86	86	60	43	35	62.1%
Claude Opus 4.6 (Reasoning)	75	67	61	57	46	61.2%
Claude Sonnet 4	100	72	61	42	20	59.0%
Qwen3 235B A22B Instruct 2507	80	61	61	60	23	57.1%
DeepSeek V4 Flash (Reasoning)	100	53	47	47	21	53.7%
DeepSeek V4 Flash	96	66	57	49	0	53.7%
GPT-5.4	80	61	38	37	34	49.9%
Claude Opus 4.6	82	62	48	35	19	49.4%
Grok 4.1 Fast	88	69	32	29	27	49.0%
Mistral NeMO	75	66	51	45	6	48.7%
MiniMax M2.5	60	51	49	37	34	46.1%
Z.AI GLM 5.1	100	62	35	21	9	45.5%
GPT-5.5	67	48	36	32	30	42.6%
Mistral Small Creative	100	58	42	10	1	42.3%
Z.AI GLM 5	81	43	32	32	21	42.0%
Ministral 3 8B	81	69	45	10	0	41.0%
Grok 4 Fast	61	53	39	33	19	40.9%
Mistral Small 4 (Reasoning)	58	58	37	29	23	40.8%
Mistral Small 3.2 24B	100	35	34	30	1	40.0%
Qwen3.6 Max Preview	82	49	37	30	0	39.5%
Llama 3.1 70B	100	43	28	17	1	37.7%
Grok 4.20	51	40	36	33	29	37.7%
DeepSeek V4 Pro	50	43	40	27	26	37.4%
Xiaomi MIMO v2.5 Pro	61	41	31	30	23	37.3%
Claude Sonnet 4.6 (Reasoning)	58	56	43	29	0	37.2%
Claude Sonnet 4.5	66	48	36	33	0	36.6%
GPT-4o, Aug. 6th (temp=1)	60	51	41	31	0	36.6%
Mistral Large	81	76	16	4	0	35.5%
Grok 4.20 (Beta)	57	43	38	38	0	35.3%
Qwen 3.5 397B A17B	54	43	36	23	13	33.8%
Claude Haiku 4.5	57	41	37	19	13	33.5%
Mistral Large 3	60	52	38	15	0	33.0%
Ministral 3 14B	71	45	31	18	0	32.9%
Qwen 3.5 Plus (2026-02-15)	89	48	26	0	0	32.5%
Qwen 3.6 Flash	55	46	23	23	16	32.5%
MiniMax M2.7	74	46	32	11	0	32.4%
GPT-4o, May 13th (temp=0)	47	46	29	27	14	32.4%
Ministral 8B	63	47	31	19	0	32.0%
Mistral Small 4	71	55	29	0	0	31.1%
Claude Sonnet 4.6	100	35	11	9	0	31.0%
Gemma 3 27B	55	43	23	20	9	30.0%
GPT-5.4 Mini	41	36	34	26	11	29.7%
Gemini 2.5 Flash Lite	49	40	27	22	9	29.5%
Hermes 3 405B	54	50	28	13	0	29.0%
GPT-5.5 (Reasoning, Low)	37	34	31	26	16	29.0%
Stealth: Hunter Alpha	50	45	38	9	1	28.6%
Z.AI GLM 5 Turbo	63	57	10	5	5	28.1%
Z.AI GLM 4.5 Air	76	36	15	12	0	27.9%
Gemma 3 12B	36	34	31	21	16	27.7%
GPT-4.1 Nano	48	33	29	25	0	27.0%
o4 Mini	71	21	19	12	11	26.7%
Stealth: Healer Alpha	85	27	17	4	0	26.6%
MoonshotAI: Kimi K2.5	46	43	32	9	0	26.1%
Arcee AI: Trinity Large (Preview)	46	41	31	12	0	25.9%
GPT-4o, May 13th (temp=1)	42	31	23	17	13	25.2%
Grok 4.20 (Reasoning)	58	45	13	9	0	25.0%
Gemma 3 4B	61	28	23	11	0	24.5%
ByteDance Seed 1.6 Flash	48	29	25	20	0	24.4%
Llama 3.1 Nemotron 70B	56	25	18	16	6	24.1%
Ministral 3 3B	93	26	2	0	0	24.1%
DeepSeek V3.2	45	41	33	0	0	23.9%
LFM2 24B	54	30	22	14	0	23.9%
GPT-5.4 (Reasoning)	50	29	16	14	10	23.7%
ByteDance Seed 2.0 Mini	65	23	17	9	0	22.6%
GPT-5.4 Nano (Reasoning, Low)	48	26	19	9	9	22.3%
Gemini 2.5 Flash	60	30	16	0	0	21.2%
Gemini 2.5 Flash Lite (Reasoning)	46	37	17	6	0	21.1%
Qwen 3 32B	90	6	4	0	0	20.1%
WizardLM 2 8x22b	40	23	19	18	0	19.8%
Claude Opus 4.7 (Reasoning)	62	27	9	1	0	19.7%
GPT-5 Nano	74	11	8	4	0	19.3%
Grok 4	34	32	12	12	5	19.1%
MoonshotAI: Kimi K2.6	37	21	19	14	3	18.9%
Qwen 3.5 Plus (2026-04-20)	55	22	8	8	0	18.5%
Qwen 3.6 35B	48	30	15	0	0	18.5%
DeepSeek V3 (2025-03-24)	38	27	15	13	0	18.3%
GPT-5.1	42	33	11	5	0	18.2%
Mistral Large 2	35	24	16	14	0	17.9%
Arcee AI: Trinity Mini	45	18	17	6	0	17.1%
Claude 3.7 Sonnet	56	20	6	1	0	16.7%
Claude 3.5 Sonnet	37	31	15	0	0	16.6%
GPT-4o, Aug. 6th (temp=0)	37	17	14	14	0	16.3%
GPT-5.5 (Reasoning)	27	24	16	14	0	16.2%
Xiaomi MIMO v2.5	32	23	14	9	1	15.7%
GPT-5.4 Nano	24	20	17	13	4	15.5%
Qwen 3.5 Flash	47	15	9	4	0	15.0%
GPT-5.4 Mini (Reasoning, Low)	28	23	11	6	0	13.6%
Z.AI GLM 4.6	40	20	3	0	0	12.6%
Ministral 3B	51	7	3	0	0	12.2%
ByteDance Seed 1.6	30	30	0	0	0	12.0%
GPT-4.1	28	17	6	5	0	11.2%
Grok 4.20 (Beta, Reasoning)	39	14	3	0	0	11.1%
GPT-5.4 Mini (Reasoning)	21	20	13	0	0	10.9%
Qwen 3.6 27B	29	20	3	0	0	10.3%
DeepSeek V4 Pro (Reasoning)	30	16	6	0	0	10.3%
Qwen 3.5 27B	25	25	0	0	0	10.0%
Gemini 2.5 Flash (Reasoning)	44	4	0	0	0	9.6%
Aion 2.0	31	14	0	0	0	9.0%
GPT-5.4 Nano (Reasoning)	20	12	6	4	2	8.8%
o4 Mini High	42	1	0	0	0	8.6%
Gemini 3 Flash (Preview, Reasoning)	32	5	4	0	0	8.3%
GPT-5 Mini	27	14	0	0	0	8.1%
Gemini 3 Pro (Preview)	15	12	8	1	0	7.2%
GPT-4o Mini (temp=1)	25	6	0	0	0	6.2%
Z.AI GLM 4.5	19	12	0	0	0	6.1%
DeepSeek V3 (2024-12-26)	16	10	1	0	0	5.4%
Gemini 3.1 Pro (Preview)	24	3	0	0	0	5.3%
Qwen 3.5 35B	27	0	0	0	0	5.3%
Grok 4.3	14	7	4	0	0	4.9%
Gemini 2.5 Pro	20	2	0	0	0	4.5%
GPT-5	10	7	4	0	0	4.4%
GPT-4.1 Mini	9	5	5	0	0	3.7%
Z.AI GLM 4.7 Flash	13	3	1	0	0	3.5%
Nemotron 3 Super	17	0	0	0	0	3.4%
Gemini 3 Flash (Preview)	14	0	0	0	0	2.7%
Gemini 3.1 Flash Lite	11	2	0	0	0	2.6%
Z.AI GLM 4.7	7	5	0	0	0	2.6%
Qwen 2.5 72B	5	3	3	0	0	2.5%
DeepSeek-V2 Chat	11	0	0	0	0	2.1%
DeepSeek V3.1	6	3	0	0	0	1.8%
Gemini 3.1 Flash Lite (Preview)	4	3	0	0	0	1.6%
ByteDance Seed 2.0 Lite	5	0	0	0	0	1.1%
Qwen 3.5 122B	5	0	0	0	0	1.1%
GPT-4o Mini (temp=0)	5	0	0	0	0	0.9%
Gemma 4 26B (Reasoning)	4	0	0	0	0	0.8%
GPT-5.2	3	0	0	0	0	0.6%
Gemma 4 31B	2	0	0	0	0	0.5%
Grok 4.3 (Reasoning)	2	0	0	0	0	0.5%
Gemma 4 31B (Reasoning)	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Gemini 3.1 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Gemma 4 26B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	94	98.7%
Writer: Palmyra X5	100	100	100	100	81	96.2%
Rocinante 12B	100	100	100	93	82	95.0%
Z.AI GLM 5	100	100	100	84	49	86.6%
Llama 3.1 8B	100	100	100	78	48	85.2%
Hermes 3 405B	100	100	93	91	36	84.1%
DeepSeek V4 Flash (Reasoning)	99	97	96	83	42	83.5%
Claude Sonnet 4.6 (Reasoning)	100	100	83	76	55	82.7%
Claude Opus 4.7 (Reasoning)	100	100	94	85	32	82.3%
Claude Opus 4.7	100	100	100	62	40	80.5%
DeepSeek V4 Flash	100	87	74	71	62	78.7%
Claude Haiku 4.5	100	90	84	69	43	77.2%
Claude Opus 4.5	92	84	75	64	64	75.9%
Hermes 3 70B	100	100	85	74	18	75.3%
Claude Sonnet 4.6	100	90	78	56	46	74.2%
Claude Sonnet 4.5	100	94	65	57	40	71.4%
GPT-5.4 (Reasoning, Low)	89	77	70	63	50	69.6%
Claude Sonnet 4	97	82	64	54	47	69.0%
Xiaomi MIMO v2.5 Pro	90	72	68	64	52	69.0%
Mistral Small 4	100	100	78	43	23	68.7%
Z.AI GLM 5.1	100	88	63	52	37	68.2%
MiniMax M2.7	100	93	49	49	48	67.7%
GPT-5.4	84	76	60	59	57	67.2%
Mistral Medium 3.1	91	88	67	48	36	66.2%
Claude Opus 4.6 (Reasoning)	83	83	57	53	49	64.9%
DeepSeek V4 Pro	100	98	97	16	13	64.8%
Cohere Command R+ (Aug. 2024)	100	75	73	51	23	64.6%
GPT-4o, Aug. 6th (temp=1)	100	83	55	42	42	64.5%
Z.AI GLM 5 Turbo	79	70	69	59	44	64.3%
Claude Opus 4.6	82	76	72	53	37	64.0%
Arcee AI: Trinity Large (Preview)	85	85	70	57	20	63.5%
WizardLM 2 8x22b	86	69	60	50	49	62.8%
DeepSeek V4 Pro (Reasoning)	77	68	65	53	51	62.5%
Qwen 3.5 397B A17B	100	67	55	45	42	61.9%
Grok 4.1 Fast	74	70	61	59	45	61.6%
GPT-4o Mini (temp=1)	96	70	53	48	38	60.9%
Mistral Large 3	72	67	65	64	28	59.2%
Gemma 3 27B	88	60	52	44	40	56.8%
Z.AI GLM 4.5 Air	100	70	54	47	11	56.2%
GPT-4.1	87	65	61	41	27	56.2%
Mistral Small 4 (Reasoning)	98	75	57	48	2	56.1%
Claude 3 Haiku	100	100	45	29	0	54.8%
MiniMax M2.5	100	71	43	30	27	54.1%
GPT-5.1	78	59	51	48	34	54.1%
Gemini 2.5 Flash Lite	70	68	53	50	27	53.9%
GPT-4.1 Nano	76	73	68	27	26	53.7%
Mistral Large	99	86	61	20	0	53.4%
Qwen 3.5 Plus (2026-04-20)	75	75	63	28	25	53.3%
Gemini 2.5 Flash Lite (Reasoning)	63	53	53	53	44	53.2%
Claude Opus 4	79	68	50	38	26	52.3%
GPT-4o, May 13th (temp=1)	75	58	57	46	23	51.7%
DeepSeek V3.2	69	57	52	41	34	50.5%
Mistral Large 2	90	53	31	31	29	46.8%
MoonshotAI: Kimi K2.5	62	54	40	38	38	46.2%
Grok 4 Fast	60	56	51	31	26	45.0%
GPT-5.5 (Reasoning, Low)	69	45	40	39	30	44.8%
Mistral Small Creative	81	65	36	28	13	44.6%
DeepSeek V3 (2025-03-24)	69	67	43	27	16	44.4%
Qwen 3 32B	75	52	47	42	0	43.3%
GPT-5.4 (Reasoning)	54	49	40	37	34	42.8%
Z.AI GLM 4.6	64	52	35	34	26	42.3%
Gemini 2.5 Flash	67	53	43	31	15	41.9%
Grok 4	67	55	32	31	24	41.7%
Z.AI GLM 4.5	60	47	47	28	26	41.6%
Stealth: Healer Alpha	69	65	45	28	0	41.4%
MoonshotAI: Kimi K2.6	75	50	38	35	8	41.2%
Grok 4.20	62	52	43	33	17	41.2%
Llama 3.1 70B	60	43	40	38	24	41.1%
GPT-5.5 (Reasoning)	48	42	42	38	34	40.6%
Ministral 3B	75	48	31	27	19	40.2%
o4 Mini High	60	52	42	28	15	39.5%
Qwen3.6 Max Preview	73	52	39	20	11	39.1%
GPT-5.5	52	52	50	36	4	38.9%
Aion 2.0	66	36	35	31	26	38.9%
GPT-5	57	50	44	26	16	38.6%
Ministral 3 14B	91	29	27	25	20	38.4%
GPT-4.1 Mini	60	52	32	30	17	38.1%
Ministral 3 3B	97	49	31	9	3	37.9%
Claude 3.5 Sonnet	68	65	56	0	0	37.7%
GPT-5.4 Mini (Reasoning, Low)	70	48	25	23	23	37.7%
Stealth: Hunter Alpha	58	45	33	26	23	36.9%
Qwen 3.6 Flash	72	45	29	27	8	36.1%
Gemini 2.5 Pro	56	55	49	11	2	34.5%
Xiaomi MIMO v2.5	65	34	29	27	16	34.3%
GPT-5.4 Nano	56	43	35	21	16	34.2%
Gemma 3 4B	75	34	31	16	15	34.1%
Llama 3.1 Nemotron 70B	57	48	37	14	13	33.8%
Z.AI GLM 4.7 Flash	57	54	23	19	13	33.2%
Grok 4.20 (Beta)	52	36	34	28	15	33.1%
Ministral 3 8B	81	66	13	0	0	32.2%
o4 Mini	63	41	31	17	7	31.7%
Claude 3.7 Sonnet	68	37	32	14	8	31.6%
LFM2 24B	40	38	34	27	20	31.6%
DeepSeek V3.1	47	41	26	23	16	30.7%
Qwen 3.6 35B	61	50	31	8	0	30.0%
Mistral Small 3.2 24B	60	45	22	22	0	29.7%
Grok 4.20 (Beta, Reasoning)	48	35	23	21	20	29.2%
ByteDance Seed 1.6 Flash	45	41	35	13	4	27.5%
Gemma 3 12B	44	34	34	23	2	27.4%
Qwen 3.5 Plus (2026-02-15)	43	36	32	13	1	25.0%
Grok 4.3	86	35	3	0	0	24.9%
Ministral 8B	53	37	31	1	0	24.6%
GPT-5.4 Nano (Reasoning, Low)	31	30	30	21	2	23.1%
GPT-5.4 Mini	37

Subject-first sentence starts

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets