Subject-first sentence starts

Test: Bad Writing Habits

Avg. Score

35.7%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Writer: Palmyra X5	83.2%	$0.011	22.0s	50%
2	Qwen3 235B A22B Instruct 2507	81.0%	$0.0011	59.2s	48%
3	Rocinante 12B	77.8%	$0.0014	38.4s	38%
4	Llama 3.1 8B	71.0%	$0.0003	1.3m	29%
5	Mistral Small 4 (Reasoning)	60.6%	$0.0022	30.2s	25%
6	GPT-5.4	70.2%	$0.049	1.4m	39%
7	Mistral Small 4	56.1%	$0.0014	18.2s	24%
8	Ministral 3 14B	50.5%	$0.0007	11.7s	25%
9	Claude Sonnet 4.5	63.6%	$0.035	38.1s	27%
10	Grok 4.1 Fast	54.6%	$0.0018	37.8s	24%
11	GPT-5.4 (Reasoning, Low)	67.7%	$0.055	1.4m	36%
12	Mistral Small Creative	49.3%	$0.0007	9.1s	22%
13	GPT-5.4 Mini	50.0%	$0.015	16.8s	25%
14	Z.AI GLM 5	58.9%	$0.0084	1.2m	26%
15	Claude Haiku 4.5	52.5%	$0.011	21.6s	21%
16	GPT-5.4 Mini (Reasoning, Low)	47.8%	$0.015	16.8s	23%
17	Claude 3.5 Haiku	51.4%	$0.0035	10.8s	17%
18	Z.AI GLM 5 Turbo	49.6%	$0.0081	33.2s	22%
19	Mistral Medium 3.1	43.3%	$0.0048	36.5s	25%
20	Hermes 3 70B	57.9%	$0.0010	1.2m	20%
21	Grok 4 Fast	45.4%	$0.0017	24.1s	20%
22	MiniMax M2.5	52.7%	$0.0034	1.3m	23%
23	Hermes 3 405B	53.5%	$0.0032	53.2s	19%
24	Grok 4.20 (Beta)	38.9%	$0.018	15.8s	25%
25	Claude Sonnet 4.6	53.3%	$0.031	39.3s	21%
26	Claude Sonnet 4	53.0%	$0.032	43.7s	22%
27	Llama 3.1 Nemotron 70B	45.5%	$0.0038	31.7s	18%
28	Arcee AI: Trinity Large (Preview)	44.0%	$0.0000	43.6s	19%
29	Llama 3.1 70B	44.4%	$0.0015	29.4s	17%
30	Claude 3 Haiku	45.6%	$0.0025	14.9s	14%
31	Claude Opus 4.5	58.8%	$0.070	53.4s	27%
32	Gemini 2.5 Flash Lite	34.5%	$0.0009	9.5s	18%
33	GPT-5.4 Mini (Reasoning)	45.0%	$0.022	28.1s	19%
34	ByteDance Seed 1.6 Flash	38.0%	$0.0013	27.3s	18%
35	Ministral 8B	37.6%	$0.0004	10.4s	15%
36	GPT-4o, Aug. 6th (temp=1)	44.8%	$0.018	24.4s	17%
37	Stealth: Healer Alpha	34.3%	$0.0000	23.7s	18%
38	MiniMax M2.7	46.3%	$0.0040	1.1m	17%
39	Ministral 3 8B	38.5%	$0.0008	19.6s	14%
40	DeepSeek V3 (2025-03-24)	42.0%	$0.0014	39.4s	14%
41	GPT-4.1	39.4%	$0.018	44.7s	21%
42	Gemma 3 12B	36.6%	$0.0004	41.3s	17%
43	Mistral Large 2	40.1%	$0.013	29.4s	16%
44	Mistral Large 3	36.9%	$0.0033	30.3s	16%
45	Stealth: Hunter Alpha	40.0%	$0.0000	55.0s	17%
46	Gemini 2.5 Flash Lite (Reasoning)	33.3%	$0.0028	30.8s	17%
47	GPT-4o Mini (temp=1)	38.9%	$0.0012	34.8s	13%
48	Gemma 3 27B	39.8%	$0.0006	52.6s	14%
49	GPT-4.1 Nano	34.2%	$0.0007	13.3s	12%
50	Claude Sonnet 4.6 (Reasoning)	53.7%	$0.060	1.2m	22%
51	Claude Opus 4.6	56.4%	$0.078	1.2m	25%
52	Claude Opus 4.6 (Reasoning)	58.5%	$0.088	1.4m	26%
53	Cohere Command R+ (Aug. 2024)	45.9%	$0.020	52.5s	13%
54	Mistral Large	37.3%	$0.014	30.9s	13%
55	GPT-5.4 Nano	28.8%	$0.0057	26.3s	16%
56	Gemini 2.5 Flash	28.1%	$0.0052	10.6s	13%
57	LFM2 24B	28.6%	$0.0002	28.4s	13%
58	GPT-5.4 Nano (Reasoning, Low)	26.5%	$0.0055	20.6s	15%
59	Gemma 3 4B	29.4%	$0.0002	20.0s	11%
60	Mistral NeMO	28.6%	$0.0005	10.1s	10%
61	GPT-5.4 Nano (Reasoning)	26.8%	$0.0061	24.5s	15%
62	Z.AI GLM 4.6	28.9%	$0.0065	51.5s	17%
63	Grok 4.20 (Beta, Reasoning)	34.7%	$0.039	34.0s	18%
64	Gemini 2.5 Flash (Reasoning)	32.2%	$0.011	21.5s	11%
65	Qwen 3 32B	32.1%	$0.0015	54.6s	13%
66	GPT-5.1	51.4%	$0.054	1.8m	22%
67	GPT-5.4 (Reasoning)	65.0%	$0.089	2.6m	29%
68	GPT-4o, May 13th (temp=1)	30.5%	$0.033	14.4s	15%
69	Ministral 3B	23.3%	$0.0001	8.1s	10%
70	o4 Mini	25.8%	$0.015	25.7s	14%
71	WizardLM 2 8x22b	38.5%	$0.0026	1.8m	16%
72	Z.AI GLM 4.5	29.0%	$0.0051	42.1s	12%
73	GPT-4.1 Mini	24.8%	$0.0027	19.0s	10%
74	DeepSeek V3.2	33.7%	$0.0014	1.9m	17%
75	Gemini 2.5 Pro	29.5%	$0.036	36.2s	16%
76	Aion 2.0	28.8%	$0.0064	1.3m	15%
77	Claude 3.7 Sonnet	32.1%	$0.042	46.7s	17%
78	Gemini 3.1 Flash Lite (Preview)	21.3%	$0.0030	8.4s	8%
79	Qwen 3.5 Plus (2026-02-15)	22.7%	$0.0060	31.5s	12%
80	o4 Mini High	28.1%	$0.025	47.2s	14%
81	MoonshotAI: Kimi K2.5	43.7%	$0.019	3.2m	22%
82	Grok 4	43.0%	$0.048	1.7m	16%
83	Ministral 3 3B	18.7%	$0.0005	11.1s	5%
84	Claude 3.5 Sonnet	32.1%	$0.048	35.5s	11%
85	DeepSeek V3.1	26.6%	$0.0020	1.8m	13%
86	Gemini 3 Flash (Preview)	17.4%	$0.0078	19.6s	7%
87	DeepSeek V3 (2024-12-26)	22.0%	$0.0021	54.6s	7%
88	DeepSeek-V2 Chat	21.7%	$0.0021	53.3s	6%
89	GPT-4o, Aug. 6th (temp=0)	17.5%	$0.023	22.7s	8%
90	GPT-4o, May 13th (temp=0)	18.6%	$0.035	14.1s	8%
91	Gemini 3 Flash (Preview, Reasoning)	17.3%	$0.012	30.1s	5%
92	Z.AI GLM 4.7 Flash	16.8%	$0.0017	1.2m	8%
93	GPT-5 Mini	17.7%	$0.0100	57.4s	7%
94	Arcee AI: Trinity Mini	11.6%	$0.0003	9.2s	0%
95	Qwen 3.5 397B A17B	33.8%	$0.014	3.0m	15%
96	Qwen 3.5 Flash	12.9%	$0.0025	47.5s	2%
97	Z.AI GLM 4.7	16.6%	$0.010	1.4m	8%
98	Qwen 2.5 72B	8.8%	$0.0010	36.7s	2%
99	GPT-4o Mini (temp=0)	9.5%	$0.0012	34.8s	0%
100	Nemotron 3 Super	13.3%	$0.0000	1.4m	5%
101	Gemini 3 Pro (Preview)	20.1%	$0.055	54.4s	9%
102	Claude Opus 4	57.5%	$0.209	1.4m	28%
103	Stealth: Aurora Alpha	0.7%	$0.0000	9.8s	0%
104	Inception Mercury 2	1.1%	$0.0032	7.0s	0%
105	GPT-5.2	22.9%	$0.056	1.5m	11%
106	ByteDance Seed 2.0 Lite	22.0%	$0.012	2.2m	6%
107	GPT-5 Nano	13.5%	$0.0042	1.4m	2%
108	Qwen 3.5 35B	14.4%	$0.018	1.0m	1%
109	Inception Mercury	1.2%	$0.011	17.6s	0%
110	Nemotron 3 Nano	7.5%	$0.0010	1.1m	0%
111	Qwen 3.5 9B	7.6%	$0.0011	1.4m	0%
112	GPT-5	29.2%	$0.065	2.8m	13%
113	Qwen 3.5 122B	8.2%	$0.025	1.1m	0%
114	Qwen 3.5 27B	6.7%	$0.020	1.6m	0%
115	ByteDance Seed 1.6	11.8%	$0.013	2.5m	0%
116	Gemini 3.1 Pro (Preview)	23.9%	$0.107	1.8m	6%
117	ByteDance Seed 2.0 Mini	22.7%	$0.0045	4.9m	9%
118	Mistral Small 3.2 24B	25.5%	$0.0069	5.7m	9%
35.67%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	96	88	80	92.7%
Rocinante 12B	100	100	98	83	78	91.8%
GPT-5.4 (Reasoning)	100	91	87	86	60	84.7%
GPT-5.4 (Reasoning, Low)	100	89	80	71	71	82.5%
Qwen3 235B A22B Instruct 2507	100	100	98	87	27	82.5%
Claude Sonnet 4.6 (Reasoning)	97	92	81	74	62	81.2%
Claude Sonnet 4.6	96	79	78	77	63	78.4%
Claude Sonnet 4.5	100	87	71	67	64	77.9%
Gemma 3 4B	100	78	76	42	39	67.1%
Hermes 3 70B	100	96	60	56	23	66.9%
GPT-5.4	81	75	67	60	51	66.8%
Claude Opus 4.5	83	81	72	65	29	65.9%
Claude Opus 4.6 (Reasoning)	100	72	71	39	35	63.2%
Llama 3.1 Nemotron 70B	85	73	63	41	35	59.2%
MiniMax M2.7	88	60	51	50	42	58.2%
Cohere Command R+ (Aug. 2024)	99	82	40	36	34	58.2%
Claude Opus 4	75	74	71	42	24	57.3%
Claude Opus 4.6	76	70	50	46	44	57.3%
WizardLM 2 8x22b	97	69	53	39	27	57.0%
Gemma 3 12B	87	73	53	41	31	56.9%
GPT-5.4 Mini	77	70	55	41	41	56.6%
Z.AI GLM 5	100	78	67	20	13	55.5%
Llama 3.1 8B	100	100	56	18	0	54.8%
GPT-5.4 Mini (Reasoning)	75	64	55	46	34	54.6%
Claude 3.5 Haiku	100	67	37	37	31	54.5%
Claude Haiku 4.5	100	61	43	36	33	54.4%
Hermes 3 405B	100	100	45	14	11	54.0%
MoonshotAI: Kimi K2.5	74	60	54	48	29	53.2%
GPT-5.4 Mini (Reasoning, Low)	69	63	56	46	30	53.1%
Aion 2.0	64	59	55	43	33	50.8%
Z.AI GLM 5 Turbo	78	67	47	43	16	50.2%
Gemini 2.5 Pro	63	61	61	45	11	48.1%
Gemini 2.5 Flash	70	48	42	41	30	46.3%
Gemma 3 27B	59	56	55	43	18	46.2%
GPT-4o Mini (temp=1)	90	45	42	38	15	46.0%
Arcee AI: Trinity Large (Preview)	68	50	47	39	22	45.0%
Gemini 2.5 Flash Lite	52	49	43	42	37	44.7%
Ministral 8B	79	60	46	23	15	44.5%
Gemini 2.5 Flash (Reasoning)	81	77	37	14	9	43.7%
Ministral 3 14B	75	50	43	28	18	43.1%
Claude Sonnet 4	68	54	49	29	11	42.1%
Stealth: Hunter Alpha	88	34	33	31	20	41.4%
DeepSeek V3 (2025-03-24)	94	56	26	26	0	40.3%
GPT-5.1	54	49	38	36	25	40.2%
DeepSeek V3.2	54	52	43	39	12	40.1%
Gemini 2.5 Flash Lite (Reasoning)	77	38	32	30	20	39.5%
LFM2 24B	86	61	22	21	0	37.9%
Mistral Small 4 (Reasoning)	55	55	44	17	15	37.3%
GPT-4o, Aug. 6th (temp=1)	75	58	29	18	0	36.2%
Grok 4.20 (Beta)	64	45	39	21	10	35.9%
Mistral Small Creative	53	46	43	17	13	34.6%
ByteDance Seed 1.6 Flash	79	34	29	23	8	34.6%
Mistral Small 3.2 24B	81	35	29	28	0	34.6%
GPT-4.1	62	53	30	27	0	34.4%
Stealth: Healer Alpha	80	40	38	14	0	34.4%
Gemini 3 Flash (Preview, Reasoning)	48	39	35	28	16	33.2%
Ministral 3B	65	60	37	0	0	32.4%
Z.AI GLM 4.6	49	41	28	23	22	32.4%
Mistral Large 2	39	37	32	29	18	31.2%
Claude 3 Haiku	73	26	21	21	14	30.9%
Llama 3.1 70B	100	31	10	7	0	29.6%
Mistral Medium 3.1	46	42	31	15	13	29.2%
GPT-5.2	42	41	37	16	5	28.2%
MiniMax M2.5	73	26	25	16	0	28.0%
Grok 4 Fast	43	31	31	21	13	28.0%
Gemini 3 Pro (Preview)	49	36	27	20	7	27.6%
Claude 3.7 Sonnet	50	40	28	20	0	27.6%
Gemini 3 Flash (Preview)	38	37	36	24	0	27.0%
Mistral Large 3	46	31	20	19	17	26.4%
GPT-4o, May 13th (temp=1)	60	26	16	14	12	25.7%
Mistral Large	48	31	27	20	1	25.4%
Grok 4.1 Fast	48	27	26	17	6	24.8%
Claude 3.5 Sonnet	70	23	14	8	7	24.1%
ByteDance Seed 2.0 Lite	51	31	23	16	0	24.1%
DeepSeek V3.1	33	32	25	20	9	24.0%
DeepSeek-V2 Chat	54	22	18	11	10	22.9%
GPT-5.4 Nano	43	28	18	14	11	22.9%
Ministral 3 8B	60	39	12	2	0	22.6%
Ministral 3 3B	45	41	13	10	0	21.8%
Qwen 3.5 397B A17B	40	31	28	8	0	21.4%
GPT-5.4 Nano (Reasoning, Low)	27	27	24	20	4	20.4%
GPT-5.4 Nano (Reasoning)	37	35	14	11	5	20.3%
Z.AI GLM 4.7	28	27	23	14	2	18.9%
ByteDance Seed 2.0 Mini	55	25	8	0	0	17.4%
o4 Mini High	46	25	9	7	0	17.4%
DeepSeek V3 (2024-12-26)	38	25	21	3	0	17.4%
Mistral Small 4	34	17	15	12	9	17.3%
GPT-5	40	18	10	9	7	16.9%
GPT-4.1 Mini	41	23	18	0	0	16.2%
Grok 4	30	28	10	6	2	15.3%
Gemini 3.1 Flash Lite (Preview)	37	17	13	5	3	15.1%
GPT-4o, Aug. 6th (temp=0)	28	23	17	4	0	14.3%
Mistral NeMO	32	27	8	0	0	13.5%
GPT-4.1 Nano	25	20	16	2	0	12.3%
Arcee AI: Trinity Mini	43	17	0	0	0	12.0%
Qwen 3.5 9B	39	18	0	0	0	11.4%
Z.AI GLM 4.7 Flash	25	15	14	0	0	10.9%
Qwen 2.5 72B	24	19	5	3	2	10.7%
Gemini 3.1 Pro (Preview)	23	16	11	3	0	10.5%
Grok 4.20 (Beta, Reasoning)	23	13	10	2	0	9.7%
Qwen 3.5 Plus (2026-02-15)	21	14	7	3	0	9.0%
Z.AI GLM 4.5	23	13	5	4	0	9.0%
Qwen 3.5 Flash	45	0	0	0	0	9.0%
GPT-4o, May 13th (temp=0)	24	16	5	0	0	8.8%
GPT-5 Mini	23	8	5	0	0	7.2%
Qwen 3.5 35B	31	2	0	0	0	6.5%
GPT-5 Nano	17	9	6	0	0	6.3%
Qwen 3 32B	18	8	4	1	0	6.3%
o4 Mini	11	10	6	2	1	6.3%
GPT-4o Mini (temp=0)	27	3	1	0	0	6.1%
Nemotron 3 Super	17	13	0	0	0	6.0%
Nemotron 3 Nano	15	7	0	0	0	4.4%
Qwen 3.5 27B	19	0	0	0	0	3.8%
ByteDance Seed 1.6	14	0	0	0	0	2.8%
Stealth: Aurora Alpha	5	0	0	0	0	1.0%
Inception Mercury 2	4	0	0	0	0	0.9%
Qwen 3.5 122B	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	85	97.0%
MiniMax M2.7	100	100	100	94	92	97.0%
Claude Sonnet 4.5	100	100	100	99	76	95.1%
Claude Opus 4.6 (Reasoning)	100	100	100	100	69	93.9%
Claude Haiku 4.5	100	100	99	94	77	93.9%
GPT-4o Mini (temp=1)	100	100	95	95	76	93.0%
Claude Opus 4	100	100	100	88	57	89.1%
Rocinante 12B	100	100	100	85	60	88.9%
Claude Sonnet 4.6	99	90	89	84	76	87.5%
Claude 3.5 Haiku	100	100	92	90	54	87.2%
GPT-5.4 (Reasoning)	97	93	82	80	79	86.2%
Mistral Small 4 (Reasoning)	100	98	89	78	62	85.3%
Claude Opus 4.6	100	100	100	100	21	84.3%
Z.AI GLM 5 Turbo	97	93	89	76	66	84.2%
GPT-4o, Aug. 6th (temp=1)	100	100	95	68	57	84.1%
GPT-5.4	100	86	81	77	74	83.8%
GPT-5.4 (Reasoning, Low)	94	85	84	77	74	82.8%
Claude 3 Haiku	100	100	90	85	29	80.7%
Claude Sonnet 4	100	100	88	59	54	80.0%
Grok 4	100	96	90	65	46	79.5%
Claude Sonnet 4.6 (Reasoning)	100	100	74	59	58	78.0%
Stealth: Hunter Alpha	90	84	83	74	56	77.5%
DeepSeek V3 (2025-03-24)	100	100	90	60	37	77.4%
Llama 3.1 8B	100	100	99	77	8	76.8%
Z.AI GLM 5	100	85	74	62	56	75.4%
WizardLM 2 8x22b	95	89	69	63	59	74.7%
Claude Opus 4.5	100	100	82	44	42	73.7%
Mistral Large 2	100	89	73	56	48	73.1%
Gemma 3 4B	99	81	81	51	50	72.4%
Z.AI GLM 4.5	94	94	81	55	31	71.0%
DeepSeek V3.2	90	72	66	66	58	70.6%
Mistral Small Creative	99	85	79	52	37	70.3%
MiniMax M2.5	92	73	68	66	51	70.1%
GPT-5.4 Mini (Reasoning, Low)	85	73	67	64	59	69.8%
Arcee AI: Trinity Large (Preview)	100	91	77	48	16	66.4%
Ministral 3 14B	96	68	66	50	46	65.3%
GPT-5.1	90	71	71	52	41	65.2%
Mistral Large 3	100	100	48	45	26	63.8%
Mistral Large	93	71	69	53	27	62.7%
Aion 2.0	81	67	62	60	43	62.7%
Ministral 8B	82	74	65	60	31	62.4%
Gemini 2.5 Flash Lite	82	74	60	41	41	59.6%
Cohere Command R+ (Aug. 2024)	100	82	72	43	0	59.5%
Mistral Medium 3.1	87	60	57	48	44	59.1%
Hermes 3 70B	100	92	62	38	1	58.5%
Gemma 3 12B	71	67	63	59	32	58.3%
GPT-5.4 Mini	76	60	51	48	48	56.8%
Grok 4.1 Fast	91	49	47	47	44	55.6%
GPT-4.1	86	59	51	38	37	54.1%
Gemini 2.5 Pro	67	60	60	49	34	54.0%
Gemini 3 Pro (Preview)	87	57	51	48	24	53.5%
MoonshotAI: Kimi K2.5	77	61	59	40	28	53.0%
Llama 3.1 70B	77	65	60	49	13	53.0%
GPT-5.4 Mini (Reasoning)	80	53	50	48	33	52.7%
Claude 3.7 Sonnet	67	64	54	47	32	52.6%
DeepSeek V3.1	92	80	39	31	21	52.6%
Gemini 2.5 Flash (Reasoning)	81	80	58	43	0	52.4%
Gemini 2.5 Flash	100	48	43	36	33	51.8%
DeepSeek V3 (2024-12-26)	100	72	47	19	18	51.2%
ByteDance Seed 1.6 Flash	63	55	52	51	35	51.1%
LFM2 24B	79	71	53	43	6	50.7%
GPT-4.1 Nano	72	56	50	39	35	50.4%
GPT-4o, May 13th (temp=1)	68	59	53	36	32	49.6%
Grok 4.20 (Beta)	55	48	48	46	45	48.4%
Grok 4 Fast	71	69	65	29	6	48.2%
Ministral 3 8B	75	48	48	37	34	48.2%
GPT-4.1 Mini	66	59	58	32	26	48.1%
Claude 3.5 Sonnet	100	48	47	41	1	47.4%
Mistral NeMO	100	59	39	35	3	47.2%
GPT-5.4 Nano (Reasoning, Low)	69	57	46	32	29	46.5%
Mistral Small 4	68	64	56	40	0	45.6%
GPT-5.4 Nano	66	49	47	44	19	45.2%
Gemini 2.5 Flash Lite (Reasoning)	85	80	35	24	0	44.9%
Hermes 3 405B	93	84	33	11	0	44.2%
Stealth: Healer Alpha	61	53	47	41	17	43.7%
Llama 3.1 Nemotron 70B	100	53	29	17	6	41.1%
DeepSeek-V2 Chat	100	64	19	13	4	39.8%
Z.AI GLM 4.6	51	48	36	32	29	39.2%
ByteDance Seed 2.0 Mini	55	43	40	29	27	38.9%
Nemotron 3 Super	73	50	46	16	9	38.8%
Z.AI GLM 4.7	74	32	32	28	28	38.7%
Qwen 3 32B	52	49	45	39	1	37.2%
GPT-5	55	46	36	33	15	37.0%
GPT-5.2	57	43	36	32	17	36.9%
Arcee AI: Trinity Mini	100	41	23	6	3	34.5%
Ministral 3 3B	69	40	25	16	14	32.6%
Ministral 3B	54	38	36	24	9	32.0%
GPT-5.4 Nano (Reasoning)	51	33	32	32	9	31.4%
Qwen 3.5 Plus (2026-02-15)	58	27	25	25	13	29.7%
Grok 4.20 (Beta, Reasoning)	55	38	18	15	12	27.4%
Gemini 3.1 Pro (Preview)	42	30	25	20	19	27.4%
Gemini 3 Flash (Preview)	39	32	28	23	9	26.3%
o4 Mini	40	34	23	16	16	25.8%
GPT-4o Mini (temp=0)	37	30	28	18	15	25.4%
Z.AI GLM 4.7 Flash	33	30	22	20	17	24.4%
o4 Mini High	39	29	25	15	11	23.7%
ByteDance Seed 1.6	25	23	23	15	12	19.3%
ByteDance Seed 2.0 Lite	64	12	10	10	0	19.3%
Gemini 3 Flash (Preview, Reasoning)	35	33	7	0	0	15.1%
GPT-5 Mini	30	12	10	10	0	12.6%
GPT-4o, May 13th (temp=0)	54	5	2	0	0	12.4%
GPT-4o, Aug. 6th (temp=0)	35	16	4	2	0	11.4%
Qwen 2.5 72B	36	9	6	6	0	11.2%
GPT-5 Nano	23	17	12	2	1	11.1%
Qwen 3.5 397B A17B	16	15	8	6	6	10.1%
Gemini 3.1 Flash Lite (Preview)	16	6	2	0	0	4.9%
Mistral Small 3.2 24B	13	1	1	0	0	3.1%
Qwen 3.5 35B	15	0	0	0	0	3.0%
Qwen 3.5 122B	13	1	0	0	0	2.9%
Qwen 3.5 Flash	9	0	0	0	0	1.8%
Nemotron 3 Nano	9	0	0	0	0	1.8%
Qwen 3.5 27B	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	78	95.7%
Qwen3 235B A22B Instruct 2507	100	100	99	82	67	89.7%
Llama 3.1 8B	100	100	100	83	40	84.5%
Claude Opus 4	100	100	100	62	50	82.5%
Rocinante 12B	100	100	100	52	2	70.8%
Mistral Small 4	95	93	68	60	13	66.1%
Mistral Small 4 (Reasoning)	79	77	68	57	9	58.1%
GPT-5.4	68	67	62	44	34	54.8%
Claude Opus 4.5	83	66	43	43	31	53.5%
Claude Haiku 4.5	100	53	46	35	34	53.4%
Claude Sonnet 4	100	56	47	37	25	53.0%
Mistral Large 2	85	67	47	34	28	52.3%
Ministral 3 8B	100	91	29	23	19	52.3%
Claude Sonnet 4.5	75	72	61	29	17	50.9%
Claude Sonnet 4.6 (Reasoning)	65	56	46	45	41	50.7%
Hermes 3 405B	88	66	45	39	0	47.5%
Hermes 3 70B	74	69	47	42	0	46.4%
Mistral Medium 3.1	68	61	55	45	0	45.8%
MiniMax M2.5	71	56	50	45	6	45.8%
Mistral Large 3	100	62	24	19	17	44.6%
Ministral 3 14B	85	58	43	31	5	44.2%
Llama 3.1 70B	82	48	41	24	17	42.4%
Claude Sonnet 4.6	74	58	55	12	0	40.0%
GPT-5.4 (Reasoning, Low)	59	50	44	28	17	39.8%
Claude Opus 4.6	57	48	48	24	23	39.8%
Mistral Large	75	45	42	27	4	38.6%
Z.AI GLM 5	54	48	44	27	15	37.6%
Llama 3.1 Nemotron 70B	69	43	41	31	0	36.9%
Grok 4.1 Fast	71	48	48	11	0	35.8%
MoonshotAI: Kimi K2.5	54	41	28	27	26	35.0%
GPT-5	97	54	8	3	0	32.4%
ByteDance Seed 1.6 Flash	55	31	30	23	23	32.3%
Qwen 3.5 397B A17B	50	45	34	29	0	31.6%
Ministral 8B	46	45	33	20	9	30.8%
Arcee AI: Trinity Large (Preview)	88	60	5	0	0	30.6%
Claude 3.5 Haiku	48	43	34	27	0	30.4%
Z.AI GLM 5 Turbo	41	37	33	22	19	30.3%
Gemma 3 12B	61	35	29	20	0	29.2%
DeepSeek V3 (2025-03-24)	55	49	31	0	0	27.1%
Gemma 3 27B	49	37	25	12	11	27.0%
MiniMax M2.7	40	34	28	24	7	26.9%
GPT-4.1	60	41	27	0	0	25.5%
GPT-4o, Aug. 6th (temp=1)	42	39	27	17	0	25.0%
Qwen 3 32B	45	36	35	6	0	24.5%
WizardLM 2 8x22b	43	39	31	3	0	23.3%
Grok 4 Fast	53	31	18	6	5	22.8%
Grok 4.20 (Beta)	47	35	16	10	0	21.5%
GPT-5.4 Nano (Reasoning, Low)	39	24	18	15	7	20.7%
GPT-4o Mini (temp=1)	43	23	23	7	6	20.6%
GPT-5.4 (Reasoning)	39	33	31	0	0	20.6%
Gemini 2.5 Flash (Reasoning)	100	0	0	0	0	20.0%
Qwen 3.5 35B	44	40	14	0	0	19.6%
Claude 3 Haiku	71	14	13	0	0	19.6%
ByteDance Seed 2.0 Lite	39	30	26	3	0	19.5%
Claude Opus 4.6 (Reasoning)	47	41	7	0	0	19.0%
Mistral Small Creative	44	37	11	2	0	19.0%
DeepSeek V3 (2024-12-26)	60	20	10	0	0	18.1%
GPT-5.1	45	25	14	3	0	17.2%
Claude 3.5 Sonnet	48	16	11	9	0	16.8%
LFM2 24B	49	16	13	5	0	16.8%
Gemma 3 4B	25	23	14	14	5	16.2%
Mistral NeMO	39	18	16	0	0	14.7%
GPT-4.1 Nano	30	22	19	2	0	14.5%
GPT-4o, May 13th (temp=1)	53	19	0	0	0	14.3%
DeepSeek-V2 Chat	38	14	11	7	0	13.8%
GPT-5.4 Nano	23	18	12	9	5	13.3%
GPT-5.4 Mini	26	20	12	7	0	13.1%
GPT-5.4 Mini (Reasoning)	43	18	0	0	0	12.2%
Claude 3.7 Sonnet	34	26	0	0	0	12.0%
Ministral 3B	25	18	13	0	0	11.1%
Gemini 2.5 Flash Lite	31	11	5	4	0	10.3%
GPT-4o, May 13th (temp=0)	31	16	3	0	0	10.2%
GPT-5 Nano	51	0	0	0	0	10.2%
Cohere Command R+ (Aug. 2024)	41	10	0	0	0	10.2%
GPT-5.4 Nano (Reasoning)	27	13	10	0	0	10.0%
GPT-4.1 Mini	28	14	5	0	0	9.6%
Mistral Small 3.2 24B	41	5	0	0	0	9.2%
GPT-5.4 Mini (Reasoning, Low)	23	16	6	0	0	9.1%
DeepSeek V3.2	23	10	6	0	0	7.9%
Z.AI GLM 4.6	23	9	6	0	0	7.7%
Z.AI GLM 4.5	36	0	0	0	0	7.3%
Ministral 3 3B	34	1	0	0	0	7.1%
ByteDance Seed 1.6	31	3	0	0	0	7.0%
Aion 2.0	19	13	1	0	0	6.6%
Qwen 3.5 27B	19	14	0	0	0	6.4%
Qwen 3.5 Plus (2026-02-15)	17	15	0	0	0	6.4%
Gemini 2.5 Flash	31	0	0	0	0	6.3%
Nemotron 3 Super	31	0	0	0	0	6.1%
o4 Mini	15	11	5	0	0	6.1%
Stealth: Healer Alpha	15	9	4	1	0	5.8%
Grok 4	26	0	0	0	0	5.1%
o4 Mini High	12	9	4	0	0	5.0%
Qwen 3.5 Flash	22	3	0	0	0	4.9%
Gemini 3 Pro (Preview)	13	11	0	0	0	4.7%
GPT-5.2	23	0	0	0	0	4.5%
Qwen 2.5 72B	17	0	0	0	0	3.4%
Arcee AI: Trinity Mini	11	6	0	0	0	3.4%
GPT-4o, Aug. 6th (temp=0)	16	0	0	0	0	3.1%
Gemini 2.5 Flash Lite (Reasoning)	7	5	3	0	0	2.9%
Gemini 3.1 Pro (Preview)	13	0	0	0	0	2.6%
GPT-4o Mini (temp=0)	9	2	0	0	0	2.2%
Gemini 2.5 Pro	10	0	0	0	0	2.1%
Z.AI GLM 4.7 Flash	10	0	0	0	0	2.1%
Grok 4.20 (Beta, Reasoning)	7	1	0	0	0	1.7%
GPT-5 Mini	7	1	1	0	0	1.6%
Gemini 3 Flash (Preview, Reasoning)	7	0	0	0	0	1.4%
Gemini 3.1 Flash Lite (Preview)	2	0	0	0	0	0.5%
ByteDance Seed 2.0 Mini	1	1	0	0	0	0.3%
Qwen 3.5 9B	1	0	0	0	0	0.2%
Stealth: Hunter Alpha	1	0	0	0	0	0.2%
Nemotron 3 Nano	0	0	0	0	0	0.1%
Qwen 3.5 122B	0	0	0	0	0	0.0%
Z.AI GLM 4.7	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
DeepSeek V3.1	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	83	96.7%
GPT-5.4 (Reasoning, Low)	100	100	100	92	73	93.0%
Claude Opus 4.6 (Reasoning)	100	100	90	86	46	84.6%
Claude Sonnet 4.5	100	100	94	80	45	83.7%
Stealth: Hunter Alpha	100	93	79	66	45	76.8%
Claude Sonnet 4.6	100	98	77	60	46	76.2%
GPT-5.1	100	96	68	67	50	76.1%
Hermes 3 405B	100	96	72	58	54	76.0%
Claude Sonnet 4	100	77	72	62	57	73.7%
Mistral Small 4 (Reasoning)	100	95	65	57	52	73.7%
GPT-5.4	100	94	65	63	45	73.4%
Rocinante 12B	100	100	88	46	31	73.0%
Grok 4.1 Fast	95	84	74	53	51	71.4%
Claude Sonnet 4.6 (Reasoning)	95	90	74	48	48	71.1%
Claude Opus 4.6	100	100	59	51	45	71.0%
Z.AI GLM 5	100	72	65	64	52	70.5%
GPT-5.4 (Reasoning)	97	72	71	63	44	69.4%
Claude Opus 4	93	82	73	61	35	68.8%
MoonshotAI: Kimi K2.5	100	70	65	63	39	67.4%
Mistral Small Creative	100	71	67	57	34	66.0%
GPT-5.4 Mini	82	73	62	62	51	66.0%
Llama 3.1 8B	100	100	60	50	16	65.0%
Mistral Small 4	100	66	57	57	41	64.2%
MiniMax M2.7	81	79	79	45	33	63.4%
Mistral Large	92	75	66	63	7	60.7%
Z.AI GLM 5 Turbo	79	66	63	60	30	59.8%
Ministral 3 14B	95	77	63	54	9	59.6%
Hermes 3 70B	100	100	65	30	0	59.0%
Claude Opus 4.5	90	90	45	39	29	58.5%
Gemma 3 27B	100	71	57	44	20	58.4%
MiniMax M2.5	81	81	67	31	24	57.1%
Claude 3.5 Haiku	92	65	55	47	16	54.9%
GPT-5.4 Mini (Reasoning)	59	59	59	51	45	54.6%
Arcee AI: Trinity Large (Preview)	87	62	56	42	24	54.2%
GPT-5.4 Mini (Reasoning, Low)	74	62	52	41	29	51.5%
Stealth: Healer Alpha	67	66	65	36	23	51.4%
Qwen 3 32B	94	60	49	39	0	48.3%
DeepSeek V3.1	85	47	39	36	32	47.9%
Mistral Medium 3.1	72	69	59	25	13	47.6%
GPT-5	84	69	35	33	12	46.4%
Claude 3.7 Sonnet	81	41	40	38	31	46.2%
WizardLM 2 8x22b	86	78	40	25	0	46.0%
Claude 3.5 Sonnet	100	57	45	19	7	45.7%
Gemma 3 4B	60	53	44	43	25	45.0%
Mistral Large 2	79	62	53	30	0	44.8%
Claude Haiku 4.5	64	59	51	48	1	44.5%
Grok 4	66	59	36	31	28	43.9%
GPT-4.1 Nano	67	57	49	43	0	43.1%
DeepSeek V3 (2025-03-24)	100	43	36	21	13	42.5%
Gemma 3 12B	88	50	48	28	0	42.5%
ByteDance Seed 1.6 Flash	100	48	23	21	20	42.3%
Grok 4.20 (Beta)	58	52	34	33	33	42.0%
Gemini 2.5 Flash Lite	78	51	43	19	17	41.6%
Qwen 3.5 397B A17B	73	67	57	11	0	41.6%
Gemini 2.5 Flash (Reasoning)	68	66	47	24	3	41.5%
Aion 2.0	66	50	43	25	15	40.0%
GPT-5.4 Nano	59	41	38	32	23	38.7%
Gemini 2.5 Flash Lite (Reasoning)	67	41	37	30	17	38.4%
GPT-4o, Aug. 6th (temp=1)	54	42	38	38	14	37.1%
Mistral Small 3.2 24B	84	55	44	0	0	36.6%
Mistral Large 3	84	34	28	20	16	36.5%
Gemini 2.5 Flash	45	45	37	34	0	32.1%
GPT-5.4 Nano (Reasoning, Low)	58	53	18	16	13	31.8%
Ministral 3 8B	59	32	28	25	11	31.1%
GPT-4o Mini (temp=1)	57	50	37	12	0	31.1%
Ministral 8B	87	42	14	5	0	29.8%
LFM2 24B	62	30	24	14	12	28.3%
GPT-4o, May 13th (temp=1)	69	34	22	15	0	27.9%
Gemini 3 Pro (Preview)	59	40	25	12	1	27.3%
o4 Mini High	79	22	18	8	0	25.6%
DeepSeek V3.2	40	38	34	15	1	25.6%
Cohere Command R+ (Aug. 2024)	63	53	4	4	0	24.9%
Mistral NeMO	42	30	22	19	11	24.9%
GPT-4.1	41	39	38	0	0	23.6%
Grok 4 Fast	48	33	23	11	1	23.2%
Claude 3 Haiku	77	11	10	9	3	22.0%
Gemini 3.1 Flash Lite (Preview)	37	26	22	16	9	21.9%
Arcee AI: Trinity Mini	75	23	9	0	0	21.2%
Z.AI GLM 4.7	61	28	12	2	0	20.7%
Z.AI GLM 4.6	52	32	11	5	2	20.5%
Ministral 3B	52	26	16	9	0	20.4%
Gemini 3 Flash (Preview)	37	31	18	13	0	19.9%
Ministral 3 3B	38	24	16	12	8	19.5%
Gemini 2.5 Pro	51	23	11	11	2	19.4%
GPT-4o, May 13th (temp=0)	44	29	19	4	0	19.1%
o4 Mini	35	30	21	7	0	18.6%
GPT-5.4 Nano (Reasoning)	44	38	4	1	0	17.5%
Gemini 3 Flash (Preview, Reasoning)	38	23	18	8	0	17.4%
Llama 3.1 70B	31	29	25	0	0	17.1%
Z.AI GLM 4.7 Flash	39	32	10	3	0	17.0%
Z.AI GLM 4.5	47	25	9	2	0	16.5%
GPT-5.2	28	26	19	9	0	16.3%
DeepSeek V3 (2024-12-26)	40	25	10	0	0	15.0%
Qwen 3.5 Plus (2026-02-15)	27	26	20	1	0	14.8%
Grok 4.20 (Beta, Reasoning)	34	21	10	8	0	14.7%
GPT-4o, Aug. 6th (temp=0)	35	21	10	0	0	13.3%
Qwen 2.5 72B	47	8	4	0	0	11.9%
Gemini 3.1 Pro (Preview)	23	17	15	3	1	11.7%
Llama 3.1 Nemotron 70B	46	5	0	0	0	10.3%
GPT-5 Mini	22	12	7	0	0	8.1%
ByteDance Seed 1.6	20	18	0	0	0	7.5%
Qwen 3.5 35B	28	0	0	0	0	5.5%
Nemotron 3 Super	22	0	0	0	0	4.4%
DeepSeek-V2 Chat	9	8	4	0	0	4.1%
Qwen 3.5 122B	16	0	0	0	0	3.3%
ByteDance Seed 2.0 Mini	12	0	0	0	0	2.4%
Qwen 3.5 27B	10	0	0	0	0	2.0%
GPT-4.1 Mini	9	0	0	0	0	1.7%
GPT-4o Mini (temp=0)	3	0	0	0	0	0.6%
ByteDance Seed 2.0 Lite	3	0	0	0	0	0.6%
Qwen 3.5 Flash	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Rocinante 12B	100	100	100	78	61	87.8%
Writer: Palmyra X5	100	96	82	76	43	79.4%
Llama 3.1 8B	100	100	66	60	57	76.7%
Hermes 3 70B	100	93	93	77	20	76.6%
Cohere Command R+ (Aug. 2024)	100	100	80	74	0	70.8%
Claude 3 Haiku	83	81	71	67	38	68.1%
Mistral Medium 3.1	86	84	74	51	37	66.4%
Claude Opus 4	98	78	54	47	43	64.0%
Claude 3.5 Haiku	100	100	65	54	0	63.8%
GPT-5.4 (Reasoning, Low)	74	68	61	58	53	62.7%
Claude Opus 4.5	86	86	60	43	35	62.1%
Claude Opus 4.6 (Reasoning)	75	67	61	57	46	61.2%
Claude Sonnet 4	100	72	61	42	20	59.0%
Qwen3 235B A22B Instruct 2507	80	61	61	60	23	57.1%
GPT-5.4	80	61	38	37	34	49.9%
Claude Opus 4.6	82	62	48	35	19	49.4%
Grok 4.1 Fast	88	69	32	29	27	49.0%
Mistral NeMO	75	66	51	45	6	48.7%
MiniMax M2.5	60	51	49	37	34	46.1%
Mistral Small Creative	100	58	42	10	1	42.3%
Z.AI GLM 5	81	43	32	32	21	42.0%
Ministral 3 8B	81	69	45	10	0	41.0%
Grok 4 Fast	61	53	39	33	19	40.9%
Mistral Small 4 (Reasoning)	58	58	37	29	23	40.8%
Mistral Small 3.2 24B	100	35	34	30	1	40.0%
Llama 3.1 70B	100	43	28	17	1	37.7%
Claude Sonnet 4.6 (Reasoning)	58	56	43	29	0	37.2%
Claude Sonnet 4.5	66	48	36	33	0	36.6%
GPT-4o, Aug. 6th (temp=1)	60	51	41	31	0	36.6%
Mistral Large	81	76	16	4	0	35.5%
Grok 4.20 (Beta)	57	43	38	38	0	35.3%
Qwen 3.5 397B A17B	54	43	36	23	13	33.8%
Claude Haiku 4.5	57	41	37	19	13	33.5%
Mistral Large 3	60	52	38	15	0	33.0%
Ministral 3 14B	71	45	31	18	0	32.9%
Qwen 3.5 Plus (2026-02-15)	89	48	26	0	0	32.5%
MiniMax M2.7	74	46	32	11	0	32.4%
GPT-4o, May 13th (temp=0)	47	46	29	27	14	32.4%
Ministral 8B	63	47	31	19	0	32.0%
Mistral Small 4	71	55	29	0	0	31.1%
Claude Sonnet 4.6	100	35	11	9	0	31.0%
Gemma 3 27B	55	43	23	20	9	30.0%
GPT-5.4 Mini	41	36	34	26	11	29.7%
Gemini 2.5 Flash Lite	49	40	27	22	9	29.5%
Hermes 3 405B	54	50	28	13	0	29.0%
Stealth: Hunter Alpha	50	45	38	9	1	28.6%
Z.AI GLM 5 Turbo	63	57	10	5	5	28.1%
Gemma 3 12B	36	34	31	21	16	27.7%
GPT-4.1 Nano	48	33	29	25	0	27.0%
o4 Mini	71	21	19	12	11	26.7%
Stealth: Healer Alpha	85	27	17	4	0	26.6%
MoonshotAI: Kimi K2.5	46	43	32	9	0	26.1%
Arcee AI: Trinity Large (Preview)	46	41	31	12	0	25.9%
GPT-4o, May 13th (temp=1)	42	31	23	17	13	25.2%
Gemma 3 4B	61	28	23	11	0	24.5%
ByteDance Seed 1.6 Flash	48	29	25	20	0	24.4%
Llama 3.1 Nemotron 70B	56	25	18	16	6	24.1%
Ministral 3 3B	93	26	2	0	0	24.1%
DeepSeek V3.2	45	41	33	0	0	23.9%
LFM2 24B	54	30	22	14	0	23.9%
GPT-5.4 (Reasoning)	50	29	16	14	10	23.7%
ByteDance Seed 2.0 Mini	65	23	17	9	0	22.6%
GPT-5.4 Nano (Reasoning, Low)	48	26	19	9	9	22.3%
Gemini 2.5 Flash	60	30	16	0	0	21.2%
Gemini 2.5 Flash Lite (Reasoning)	46	37	17	6	0	21.1%
Qwen 3 32B	90	6	4	0	0	20.1%
WizardLM 2 8x22b	40	23	19	18	0	19.8%
GPT-5 Nano	74	11	8	4	0	19.3%
Grok 4	34	32	12	12	5	19.1%
DeepSeek V3 (2025-03-24)	38	27	15	13	0	18.3%
GPT-5.1	42	33	11	5	0	18.2%
Mistral Large 2	35	24	16	14	0	17.9%
Arcee AI: Trinity Mini	45	18	17	6	0	17.1%
Claude 3.7 Sonnet	56	20	6	1	0	16.7%
Claude 3.5 Sonnet	37	31	15	0	0	16.6%
GPT-4o, Aug. 6th (temp=0)	37	17	14	14	0	16.3%
GPT-5.4 Nano	24	20	17	13	4	15.5%
Qwen 3.5 Flash	47	15	9	4	0	15.0%
GPT-5.4 Mini (Reasoning, Low)	28	23	11	6	0	13.6%
Z.AI GLM 4.6	40	20	3	0	0	12.6%
Ministral 3B	51	7	3	0	0	12.2%
ByteDance Seed 1.6	30	30	0	0	0	12.0%
GPT-4.1	28	17	6	5	0	11.2%
Grok 4.20 (Beta, Reasoning)	39	14	3	0	0	11.1%
GPT-5.4 Mini (Reasoning)	21	20	13	0	0	10.9%
Qwen 3.5 27B	25	25	0	0	0	10.0%
Gemini 2.5 Flash (Reasoning)	44	4	0	0	0	9.6%
Aion 2.0	31	14	0	0	0	9.0%
GPT-5.4 Nano (Reasoning)	20	12	6	4	2	8.8%
o4 Mini High	42	1	0	0	0	8.6%
Gemini 3 Flash (Preview, Reasoning)	32	5	4	0	0	8.3%
GPT-5 Mini	27	14	0	0	0	8.1%
Gemini 3 Pro (Preview)	15	12	8	1	0	7.2%
GPT-4o Mini (temp=1)	25	6	0	0	0	6.2%
Z.AI GLM 4.5	19	12	0	0	0	6.1%
DeepSeek V3 (2024-12-26)	16	10	1	0	0	5.4%
Gemini 3.1 Pro (Preview)	24	3	0	0	0	5.3%
Qwen 3.5 35B	27	0	0	0	0	5.3%
Gemini 2.5 Pro	20	2	0	0	0	4.5%
GPT-5	10	7	4	0	0	4.4%
GPT-4.1 Mini	9	5	5	0	0	3.7%
Z.AI GLM 4.7 Flash	13	3	1	0	0	3.5%
Nemotron 3 Super	17	0	0	0	0	3.4%
Gemini 3 Flash (Preview)	14	0	0	0	0	2.7%
Z.AI GLM 4.7	7	5	0	0	0	2.6%
Qwen 2.5 72B	5	3	3	0	0	2.5%
DeepSeek-V2 Chat	11	0	0	0	0	2.1%
DeepSeek V3.1	6	3	0	0	0	1.8%
Gemini 3.1 Flash Lite (Preview)	4	3	0	0	0	1.6%
ByteDance Seed 2.0 Lite	5	0	0	0	0	1.1%
Qwen 3.5 122B	5	0	0	0	0	1.1%
GPT-4o Mini (temp=0)	5	0	0	0	0	0.9%
GPT-5.2	3	0	0	0	0	0.6%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	94	98.7%
Writer: Palmyra X5	100	100	100	100	81	96.2%
Rocinante 12B	100	100	100	93	82	95.0%
Z.AI GLM 5	100	100	100	84	49	86.6%
Llama 3.1 8B	100	100	100	78	48	85.2%
Hermes 3 405B	100	100	93	91	36	84.1%
Claude Sonnet 4.6 (Reasoning)	100	100	83	76	55	82.7%
Claude Haiku 4.5	100	90	84	69	43	77.2%
Claude Opus 4.5	92	84	75	64	64	75.9%
Hermes 3 70B	100	100	85	74	18	75.3%
Claude Sonnet 4.6	100	90	78	56	46	74.2%
Claude Sonnet 4.5	100	94	65	57	40	71.4%
Claude 3.5 Haiku	100	90	71	48	43	70.6%
GPT-5.4 (Reasoning, Low)	89	77	70	63	50	69.6%
Claude Sonnet 4	97	82	64	54	47	69.0%
Mistral Small 4	100	100	78	43	23	68.7%
MiniMax M2.7	100	93	49	49	48	67.7%
GPT-5.4	84	76	60	59	57	67.2%
Mistral Medium 3.1	91	88	67	48	36	66.2%
Claude Opus 4.6 (Reasoning)	83	83	57	53	49	64.9%
Cohere Command R+ (Aug. 2024)	100	75	73	51	23	64.6%
GPT-4o, Aug. 6th (temp=1)	100	83	55	42	42	64.5%
Z.AI GLM 5 Turbo	79	70	69	59	44	64.3%
Claude Opus 4.6	82	76	72	53	37	64.0%
Arcee AI: Trinity Large (Preview)	85	85	70	57	20	63.5%
WizardLM 2 8x22b	86	69	60	50	49	62.8%
Qwen 3.5 397B A17B	100	67	55	45	42	61.9%
Grok 4.1 Fast	74	70	61	59	45	61.6%
GPT-4o Mini (temp=1)	96	70	53	48	38	60.9%
Mistral Large 3	72	67	65	64	28	59.2%
Gemma 3 27B	88	60	52	44	40	56.8%
GPT-4.1	87	65	61	41	27	56.2%
Mistral Small 4 (Reasoning)	98	75	57	48	2	56.1%
Claude 3 Haiku	100	100	45	29	0	54.8%
MiniMax M2.5	100	71	43	30	27	54.1%
GPT-5.1	78	59	51	48	34	54.1%
Gemini 2.5 Flash Lite	70	68	53	50	27	53.9%
GPT-4.1 Nano	76	73	68	27	26	53.7%
Mistral Large	99	86	61	20	0	53.4%
Gemini 2.5 Flash Lite (Reasoning)	63	53	53	53	44	53.2%
Claude Opus 4	79	68	50	38	26	52.3%
GPT-4o, May 13th (temp=1)	75	58	57	46	23	51.7%
DeepSeek V3.2	69	57	52	41	34	50.5%
Mistral Large 2	90	53	31	31	29	46.8%
MoonshotAI: Kimi K2.5	62	54	40	38	38	46.2%
Grok 4 Fast	60	56	51	31	26	45.0%
Mistral Small Creative	81	65	36	28	13	44.6%
DeepSeek V3 (2025-03-24)	69	67	43	27	16	44.4%
Qwen 3 32B	75	52	47	42	0	43.3%
GPT-5.4 (Reasoning)	54	49	40	37	34	42.8%
Z.AI GLM 4.6	64	52	35	34	26	42.3%
Gemini 2.5 Flash	67	53	43	31	15	41.9%
Grok 4	67	55	32	31	24	41.7%
Z.AI GLM 4.5	60	47	47	28	26	41.6%
Stealth: Healer Alpha	69	65	45	28	0	41.4%
Llama 3.1 70B	60	43	40	38	24	41.1%
Ministral 3B	75	48	31	27	19	40.2%
o4 Mini High	60	52	42	28	15	39.5%
Aion 2.0	66	36	35	31	26	38.9%
GPT-5	57	50	44	26	16	38.6%
Ministral 3 14B	91	29	27	25	20	38.4%
GPT-4.1 Mini	60	52	32	30	17	38.1%
Ministral 3 3B	97	49	31	9	3	37.9%
Claude 3.5 Sonnet	68	65	56	0	0	37.7%
GPT-5.4 Mini (Reasoning, Low)	70	48	25	23	23	37.7%
Stealth: Hunter Alpha	58	45	33	26	23	36.9%
Gemini 2.5 Pro	56	55	49	11	2	34.5%
GPT-5.4 Nano	56	43	35	21	16	34.2%
Gemma 3 4B	75	34	31	16	15	34.1%
Llama 3.1 Nemotron 70B	57	48	37	14	13	33.8%
Z.AI GLM 4.7 Flash	57	54	23	19	13	33.2%
Grok 4.20 (Beta)	52	36	34	28	15	33.1%
Ministral 3 8B	81	66	13	0	0	32.2%
o4 Mini	63	41	31	17	7	31.7%
Claude 3.7 Sonnet	68	37	32	14	8	31.6%
LFM2 24B	40	38	34	27	20	31.6%
DeepSeek V3.1	47	41	26	23	16	30.7%
Mistral Small 3.2 24B	60	45	22	22	0	29.7%
Grok 4.20 (Beta, Reasoning)	48	35	23	21	20	29.2%
ByteDance Seed 1.6 Flash	45	41	35	13	4	27.5%
Gemma 3 12B	44	34	34	23	2	27.4%
Qwen 3.5 Plus (2026-02-15)	43	36	32	13	1	25.0%
Ministral 8B	53	37	31	1	0	24.6%
GPT-5.4 Nano (Reasoning, Low)	31	30	30	21	2	23.1%
GPT-5.4 Mini	37	25	22	16	15	23.1%
GPT-5.4 Nano (Reasoning)	40	33	22	16	0	21.9%
Gemini 3.1 Pro (Preview)	33	31	21	13	10	21.7%
DeepSeek-V2 Chat	42	25	22	19	0	21.6%
Mistral NeMO	43	40	21	0	0	20.9%
Gemini 3 Pro (Preview)	34	21	20	18	0	18.6%
GPT-4o, Aug. 6th (temp=0)	59	18	15	0	0	18.3%
Gemini 2.5 Flash (Reasoning)	25	24	22	13	0	16.8%
GPT-5.4 Mini (Reasoning)	34	29	13	5	1	16.5%
Nemotron 3 Super	29	24	15	0	0	13.7%
Qwen 3.5 9B	67	0	0	0	0	13.4%
Z.AI GLM 4.7	20	17	16	13	0	13.2%
Gemini 3 Flash (Preview, Reasoning)	25	19	19	2	0	12.8%
GPT-4o Mini (temp=0)	25	16	15	0	0	11.4%
ByteDance Seed 2.0 Mini	28	20	4	0	0	10.4%
DeepSeek V3 (2024-12-26)	21	9	9	4	0	8.7%
Qwen 2.5 72B	19	16	6	0	0	8.1%
GPT-5.2	30	7	0	0	0	7.4%
GPT-5 Nano	12	10	9	4	0	6.9%
Gemini 3 Flash (Preview)	18	7	5	3	0	6.4%
Arcee AI: Trinity Mini	13	10	1	0	0	4.7%
ByteDance Seed 1.6	17	5	1	0	0	4.7%
ByteDance Seed 2.0 Lite	16	4	0	0	0	4.0%
GPT-5 Mini	18	2	0	0	0	3.9%
Gemini 3.1 Flash Lite (Preview)	7	6	0	0	0	2.6%
GPT-4o, May 13th (temp=0)	9	3	0	0	0	2.4%
Nemotron 3 Nano	7	0	0	0	0	1.4%
Qwen 3.5 Flash	2	0	0	0	0	0.4%
Qwen 3.5 122B	0	0	0	0	0	0.0%
Qwen 3.5 27B	0	0	0	0	0	0.0%
Qwen 3.5 35B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning, Low)	100	90	89	88	79	89.3%
GPT-5.4 (Reasoning)	100	94	86	85	77	88.4%
GPT-5.4 Mini (Reasoning, Low)	100	91	84	83	71	85.8%
Rocinante 12B	100	100	100	64	50	82.7%
GPT-5.4 Mini (Reasoning)	88	86	79	76	76	81.2%
Llama 3.1 8B	100	100	98	62	38	79.6%
Llama 3.1 Nemotron 70B	100	100	91	51	49	78.2%
GPT-5.4	88	75	75	71	63	74.4%
GPT-5.4 Mini	84	81	66	66	66	72.6%
GPT-5.1	100	85	63	57	52	71.5%
Writer: Palmyra X5	100	88	63	59	44	70.9%
Llama 3.1 70B	89	82	78	48	43	68.0%
Mistral Small 4	98	75	72	45	25	62.8%
WizardLM 2 8x22b	87	82	53	53	38	62.5%
GPT-4o Mini (temp=1)	93	65	64	50	30	60.4%
GPT-4o, Aug. 6th (temp=1)	93	76	66	48	14	59.5%
MiniMax M2.5	91	70	52	48	35	59.2%
Claude Sonnet 4.5	79	68	56	52	41	59.2%
Grok 4 Fast	90	77	52	40	37	59.1%
Qwen 3.5 397B A17B	85	78	69	38	25	59.0%
Gemini 3 Flash (Preview, Reasoning)	83	65	57	46	40	58.4%
Gemini 3.1 Pro (Preview)	78	74	54	50	36	58.3%
Claude Sonnet 4.6 (Reasoning)	70	68	61	41	41	56.5%
Gemini 3.1 Flash Lite (Preview)	76	54	53	50	49	56.4%
GPT-4.1	65	63	59	55	39	56.2%
Qwen3 235B A22B Instruct 2507	92	63	58	53	12	55.5%
Claude Sonnet 4.6	78	64	49	47	38	55.1%
Hermes 3 405B	73	72	51	50	28	55.0%
ByteDance Seed 1.6 Flash	100	77	65	29	0	54.4%
DeepSeek V3 (2025-03-24)	100	50	45	43	34	54.4%
Z.AI GLM 4.5	78	64	51	41	38	54.4%
Claude Opus 4.6 (Reasoning)	66	66	54	48	34	53.8%
Z.AI GLM 5 Turbo	74	67	47	47	17	50.5%
Claude Sonnet 4	81	63	37	34	31	49.4%
DeepSeek V3.2	64	62	58	44	16	48.9%
Grok 4.20 (Beta)	70	53	43	42	34	48.3%
GPT-4o, May 13th (temp=1)	87	46	38	38	32	48.1%
GPT-5.4 Nano	58	56	51	50	24	47.6%
Gemini 2.5 Flash (Reasoning)	68	59	49	40	21	47.6%
Gemini 2.5 Pro	68	49	43	42	34	47.1%
GPT-5 Mini	56	46	45	44	40	46.3%
Claude Opus 4.6	72	71	47	31	9	46.1%
Gemini 3 Flash (Preview)	79	66	37	24	23	45.9%
Cohere Command R+ (Aug. 2024)	70	60	48	43	8	45.8%
Qwen 3.5 Plus (2026-02-15)	66	59	39	31	30	45.0%
Grok 4	69	50	43	42	21	45.0%
Z.AI GLM 5	82	47	46	31	15	44.1%
Grok 4.1 Fast	64	45	43	40	27	44.0%
Grok 4.20 (Beta, Reasoning)	53	52	51	48	16	43.9%
GPT-5.2	52	51	46	38	30	43.2%
Claude 3.7 Sonnet	64	54	54	44	0	43.2%
GPT-5	63	45	39	35	32	42.8%
GPT-5.4 Nano (Reasoning)	50	47	45	44	27	42.6%
MoonshotAI: Kimi K2.5	75	43	36	32	26	42.5%
Hermes 3 70B	53	43	40	38	34	41.7%
Claude 3 Haiku	66	53	53	30	0	40.4%
Stealth: Hunter Alpha	57	44	38	37	25	40.2%
ByteDance Seed 2.0 Lite	69	62	43	25	0	40.0%
Mistral Small 3.2 24B	100	52	42	0	0	38.8%
Mistral Medium 3.1	50	41	36	36	26	37.8%
DeepSeek-V2 Chat	74	57	25	23	7	37.1%
Claude Haiku 4.5	64	62	30	16	9	36.1%
Ministral 3 14B	68	54	32	16	10	35.9%
GPT-4.1 Nano	68	63	37	6	0	35.0%
LFM2 24B	59	37	34	28	16	34.7%
GPT-5.4 Nano (Reasoning, Low)	41	39	39	28	26	34.5%
Stealth: Healer Alpha	50	43	31	27	16	33.5%
GPT-4.1 Mini	70	36	29	28	0	32.7%
Claude Opus 4	69	30	23	23	19	32.7%
Mistral Large 3	41	39	28	26	25	31.9%
Gemini 2.5 Flash Lite (Reasoning)	59	39	23	23	17	31.9%
Qwen 3 32B	57	50	39	11	0	31.6%
GPT-4o, May 13th (temp=0)	55	42	28	18	13	31.2%
Gemma 3 12B	46	39	39	31	1	31.0%
Z.AI GLM 4.6	64	30	29	20	11	30.8%
DeepSeek V3 (2024-12-26)	53	52	27	11	11	30.7%
Gemini 2.5 Flash	57	45	19	16	15	30.5%
Qwen 3.5 35B	55	51	27	16	0	29.7%
Claude Opus 4.5	54	52	30	11	0	29.3%
ByteDance Seed 1.6	63	60	23	0	0	29.0%
Gemini 3 Pro (Preview)	62	38	29	14	0	28.6%
Gemma 3 4B	46	43	26	23	3	28.2%
Mistral Large 2	49	45	31	8	7	28.0%
o4 Mini	51	47	17	16	7	27.7%
Z.AI GLM 4.7	42	31	28	26	11	27.7%
Nemotron 3 Super	40	40	24	24	9	27.5%
Aion 2.0	59	33	26	19	0	27.4%
Mistral Small 4 (Reasoning)	52	30	28	17	8	26.9%
GPT-4o Mini (temp=0)	36	32	29	23	12	26.8%
GPT-4o, Aug. 6th (temp=0)	42	38	31	19	4	26.7%
Claude 3.5 Sonnet	79	31	18	4	1	26.6%
Mistral Large	36	35	24	23	12	26.1%
Mistral NeMO	42	35	19	16	13	25.0%
DeepSeek V3.1	43	42	22	9	8	24.9%
o4 Mini High	32	31	28	22	11	24.6%
Gemma 3 27B	36	33	28	21	4	24.5%
Nemotron 3 Nano	36	30	29	19	4	23.9%
ByteDance Seed 2.0 Mini	45	41	23	10	0	23.9%
Claude 3.5 Haiku	51	28	25	16	0	23.8%
Mistral Small Creative	46	37	16	10	9	23.6%
Qwen 3.5 122B	57	47	12	0	0	23.2%
Qwen 3.5 Flash	42	37	23	3	0	21.2%
Gemini 2.5 Flash Lite	31	25	19	16	13	20.9%
Ministral 8B	42	32	24	0	0	19.5%
Z.AI GLM 4.7 Flash	32	23	15	14	6	18.0%
Ministral 3B	43	34	11	1	0	17.9%
Qwen 2.5 72B	35	22	16	11	0	16.8%
Ministral 3 8B	41	30	4	0	0	14.9%
Arcee AI: Trinity Large (Preview)	37	26	0	0	0	12.7%
Qwen 3.5 9B	35	7	7	0	0	9.9%
Inception Mercury 2	18	14	13	0	0	8.9%
MiniMax M2.7	16	9	8	0	0	6.6%
Qwen 3.5 27B	20	2	0	0	0	4.4%
Ministral 3 3B	14	0	0	0	0	2.8%
GPT-5 Nano	5	5	3	0	0	2.7%
Arcee AI: Trinity Mini	12	0	0	0	0	2.3%
Inception Mercury	4	0	0	0	0	0.8%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	96	99.2%
GPT-5.4 (Reasoning, Low)	100	100	100	97	91	97.5%
GPT-5.1	100	100	100	98	86	96.8%
Grok 4.1 Fast	100	100	100	98	62	92.0%
Claude Opus 4.6 (Reasoning)	100	100	98	81	81	91.8%
Rocinante 12B	100	100	100	81	75	91.3%
Claude Sonnet 4.5	100	98	86	85	74	88.6%
Claude Sonnet 4.6 (Reasoning)	100	100	95	73	68	87.3%
GPT-5.4 Mini (Reasoning)	100	100	86	83	64	86.5%
Claude Opus 4.5	100	100	100	79	52	86.2%
Claude Sonnet 4	100	95	90	76	65	85.3%
MiniMax M2.5	100	100	100	88	30	83.6%
Claude Haiku 4.5	100	97	90	79	51	83.5%
GPT-4o, Aug. 6th (temp=1)	100	100	97	76	43	83.3%
Claude Opus 4.6	100	96	75	74	71	83.3%
Mistral Small 4 (Reasoning)	100	100	84	83	49	83.0%
DeepSeek V3 (2025-03-24)	100	90	87	87	49	82.7%
GPT-4o Mini (temp=1)	92	85	85	76	73	82.3%
Z.AI GLM 5	100	100	88	67	55	82.0%
Z.AI GLM 5 Turbo	100	84	81	73	65	80.6%
GPT-5.4 Mini (Reasoning, Low)	100	100	79	77	42	79.8%
GPT-5.4 Mini	94	86	84	73	62	79.8%
Claude 3.5 Haiku	100	100	100	60	34	78.8%
DeepSeek V3.2	91	89	80	68	65	78.8%
Mistral Small 4	90	82	74	71	70	77.4%
Mistral Small Creative	97	91	81	79	33	76.1%
Grok 4	100	90	89	63	37	76.0%
Llama 3.1 8B	100	100	100	47	27	74.8%
Ministral 3 8B	100	100	66	59	42	73.5%
Gemma 3 27B	82	81	74	65	65	73.4%
LFM2 24B	100	84	83	63	37	73.4%
GPT-5.4 Nano (Reasoning)	100	79	68	62	56	73.0%
GPT-5.2	83	77	76	66	63	72.8%
Grok 4 Fast	100	92	72	70	25	71.7%
Gemma 3 12B	79	78	71	64	61	70.4%
Claude 3.5 Sonnet	100	93	60	56	40	70.0%
GPT-4.1 Nano	100	100	76	39	34	69.6%
Gemini 2.5 Flash	100	72	65	57	48	68.2%
Arcee AI: Trinity Large (Preview)	100	97	58	53	29	67.5%
Z.AI GLM 4.5	79	69	64	64	62	67.4%
GPT-4.1	97	90	70	48	31	67.3%
Stealth: Healer Alpha	85	82	80	46	38	66.3%
Claude Sonnet 4.6	84	69	63	59	55	66.0%
Ministral 3 14B	87	85	55	53	49	65.8%
Qwen 3 32B	100	90	57	55	23	65.0%
Cohere Command R+ (Aug. 2024)	100	77	64	54	27	64.3%
Grok 4.20 (Beta)	86	76	59	58	41	64.0%
Aion 2.0	90	65	60	52	52	63.8%
Gemma 3 4B	83	65	57	55	54	63.0%
Ministral 8B	90	82	72	46	25	62.9%
GPT-5.4 Nano (Reasoning, Low)	73	69	67	62	42	62.5%
MiniMax M2.7	100	91	51	39	29	62.1%
Hermes 3 70B	94	87	57	36	34	61.7%
Ministral 3B	100	97	77	31	0	60.9%
Gemini 2.5 Flash (Reasoning)	91	78	62	48	25	60.9%
Claude Opus 4	78	77	60	54	26	59.1%
GPT-5.4 Nano	79	60	52	52	46	57.6%
Qwen 3.5 Plus (2026-02-15)	73	65	54	49	46	57.4%
Hermes 3 405B	100	85	58	43	0	57.3%
GPT-5	71	63	62	54	33	56.8%
Gemini 3 Flash (Preview, Reasoning)	74	71	58	52	28	56.5%
ByteDance Seed 1.6 Flash	100	75	56	28	22	56.2%
GPT-4o, May 13th (temp=1)	84	78	59	33	28	56.2%
MoonshotAI: Kimi K2.5	83	61	60	50	26	56.1%
GPT-5 Mini	61	59	55	55	49	55.9%
Llama 3.1 Nemotron 70B	100	75	52	51	0	55.6%
Grok 4.20 (Beta, Reasoning)	86	61	45	44	42	55.5%
Gemini 2.5 Pro	88	55	51	49	31	54.8%
Claude 3 Haiku	100	91	31	28	18	53.7%
Gemini 3.1 Pro (Preview)	89	72	40	37	30	53.6%
Z.AI GLM 4.7	84	65	54	38	26	53.5%
Claude 3.7 Sonnet	80	56	53	41	36	53.3%
Gemini 2.5 Flash Lite (Reasoning)	67	62	56	51	30	53.3%
DeepSeek-V2 Chat	100	74	63	20	6	52.6%
Mistral Medium 3.1	65	61	49	47	34	51.4%
Mistral NeMO	95	79	50	32	0	51.3%
Gemini 2.5 Flash Lite	63	60	53	41	38	51.2%
ByteDance Seed 2.0 Mini	76	74	67	18	18	50.6%
Mistral Large	76	58	57	49	8	49.4%
GPT-4.1 Mini	59	55	53	41	36	48.8%
Mistral Large 2	62	54	48	41	35	47.8%
Stealth: Hunter Alpha	70	63	54	35	14	47.2%
DeepSeek V3.1	52	51	45	42	40	46.0%
Gemini 3 Pro (Preview)	82	53	50	33	13	45.9%
Llama 3.1 70B	100	40	38	34	17	45.6%
Gemini 3 Flash (Preview)	73	64	44	25	19	45.0%
o4 Mini High	72	70	38	26	17	44.6%
Qwen 3.5 397B A17B	57	53	49	31	31	44.2%
o4 Mini	57	56	37	29	26	41.0%
Ministral 3 3B	74	71	36	16	2	39.8%
Z.AI GLM 4.7 Flash	51	47	43	32	24	39.5%
Mistral Large 3	65	46	38	36	3	37.5%
GPT-4o Mini (temp=0)	64	56	36	18	13	37.5%
DeepSeek V3 (2024-12-26)	100	49	31	6	0	37.3%
Z.AI GLM 4.6	58	47	34	24	16	35.7%
Nemotron 3 Super	56	40	34	28	17	35.0%
ByteDance Seed 2.0 Lite	72	37	36	25	0	34.2%
WizardLM 2 8x22b	69	45	30	15	11	33.9%
Qwen 3.5 Flash	73	39	35	16	5	33.8%
GPT-4o, May 13th (temp=0)	58	48	29	10	9	30.9%
Arcee AI: Trinity Mini	54	42	38	12	8	30.8%
Mistral Small 3.2 24B	48	47	44	6	0	28.9%
GPT-5 Nano	35	35	31	18	14	26.5%
ByteDance Seed 1.6	42	35	31	18	0	25.5%
Gemini 3.1 Flash Lite (Preview)	46	43	25	9	5	25.5%
Qwen 3.5 122B	45	37	21	14	7	24.8%
GPT-4o, Aug. 6th (temp=0)	66	20	12	9	3	22.0%
Qwen 3.5 35B	38	24	24	12	5	20.4%
Qwen 3.5 9B	36	26	19	13	0	18.8%
Qwen 3.5 27B	36	26	10	4	0	15.2%
Nemotron 3 Nano	24	18	11	9	1	12.5%
Qwen 2.5 72B	39	1	0	0	0	7.9%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Llama 3.1 8B	100	100	100	97	56	90.7%
Qwen3 235B A22B Instruct 2507	100	92	81	65	54	78.5%
Mistral Large	97	90	82	68	32	74.1%
Mistral Small 4	92	87	85	59	41	72.7%
GPT-5.4	90	85	74	64	31	68.9%
Writer: Palmyra X5	100	96	57	52	37	68.3%
Grok 4.20 (Beta, Reasoning)	89	74	68	62	46	67.9%
GPT-5.4 Mini	79	76	69	65	47	67.4%
GPT-5.4 Mini (Reasoning, Low)	78	69	67	56	54	64.9%
Llama 3.1 Nemotron 70B	85	80	54	53	51	64.4%
Mistral Large 2	100	73	70	52	27	64.3%
Grok 4 Fast	87	81	66	49	35	63.4%
Llama 3.1 70B	100	100	63	40	12	63.2%
GPT-5.4 (Reasoning)	75	68	63	58	51	62.9%
GPT-5.4 (Reasoning, Low)	84	61	60	58	47	62.0%
GPT-5.4 Mini (Reasoning)	66	63	61	57	56	60.7%
Claude 3.5 Haiku	100	73	64	38	26	60.1%
MiniMax M2.5	88	68	58	55	19	57.8%
Mistral Small Creative	81	71	50	47	36	57.2%
Ministral 3 14B	72	67	60	51	19	53.7%
Grok 4	74	57	52	44	38	53.0%
GPT-5.1	65	61	61	50	23	52.1%
Claude Opus 4	72	65	58	46	18	51.7%
Grok 4.1 Fast	92	53	40	39	34	51.6%
DeepSeek V3 (2025-03-24)	71	66	43	42	36	51.6%
Ministral 8B	89	76	41	26	21	50.6%
Mistral Medium 3.1	85	65	54	34	14	50.5%
Claude Sonnet 4.5	100	60	53	22	17	50.3%
Grok 4.20 (Beta)	71	64	60	34	20	49.8%
Claude 3 Haiku	100	70	33	29	11	48.5%
Rocinante 12B	100	77	46	17	1	48.3%
Mistral Small 4 (Reasoning)	100	63	42	34	0	47.9%
Claude Opus 4.5	83	75	45	35	2	47.7%
GPT-5.4 Nano (Reasoning)	69	57	47	35	29	47.3%
Hermes 3 70B	60	48	47	38	35	45.5%
Hermes 3 405B	100	40	34	25	25	44.6%
Ministral 3 8B	100	42	34	27	15	43.5%
Gemini 3.1 Pro (Preview)	86	56	38	30	3	42.6%
MoonshotAI: Kimi K2.5	76	72	27	18	17	42.1%
Claude Opus 4.6	60	42	41	39	28	41.9%
Z.AI GLM 5 Turbo	88	53	42	17	0	40.2%
Claude Haiku 4.5	78	50	35	24	8	39.1%
GPT-5 Mini	60	46	41	27	19	38.6%
Qwen 3.5 397B A17B	48	47	47	43	8	38.5%
Z.AI GLM 5	57	51	49	21	14	38.3%
Qwen 3 32B	91	53	27	9	0	35.9%
o4 Mini	47	47	46	33	5	35.6%
ByteDance Seed 1.6 Flash	53	42	37	27	19	35.5%
Arcee AI: Trinity Large (Preview)	100	40	24	5	3	34.4%
GPT-5.4 Nano (Reasoning, Low)	50	38	31	28	23	33.8%
LFM2 24B	51	43	34	31	8	33.6%
Claude Sonnet 4.6	69	28	27	21	20	33.3%
WizardLM 2 8x22b	100	40	14	9	0	32.7%
Claude Sonnet 4	49	44	40	29	0	32.4%
Gemma 3 12B	53	50	35	15	8	32.3%
GPT-4.1	51	43	37	16	9	31.4%
DeepSeek V3 (2024-12-26)	54	38	36	20	0	29.8%
ByteDance Seed 2.0 Mini	100	30	19	0	0	29.8%
Claude Sonnet 4.6 (Reasoning)	63	36	28	16	4	29.3%
Mistral NeMO	58	25	20	18	17	27.7%
Z.AI GLM 4.5	35	35	25	21	18	27.0%
Nemotron 3 Nano	50	48	30	8	0	26.9%
Ministral 3B	54	34	21	19	7	26.7%
GPT-5.2	48	27	24	20	15	26.6%
GPT-4o Mini (temp=1)	34	27	27	23	21	26.1%
GPT-4o, Aug. 6th (temp=1)	43	36	26	12	11	25.6%
Mistral Large 3	61	41	25	1	0	25.6%
o4 Mini High	56	26	25	11	5	24.4%
Claude Opus 4.6 (Reasoning)	34	30	28	20	7	23.8%
Cohere Command R+ (Aug. 2024)	64	37	11	7	0	23.8%
Stealth: Hunter Alpha	42	27	22	21	0	22.4%
MiniMax M2.7	45	33	20	12	0	21.9%
GPT-4o, May 13th (temp=0)	46	25	21	16	0	21.4%
GPT-5.4 Nano	33	26	21	19	5	20.8%
Gemini 2.5 Flash Lite (Reasoning)	33	29	24	11	0	19.3%
Mistral Small 3.2 24B	46	41	3	3	0	18.6%
Gemini 2.5 Flash (Reasoning)	56	22	8	0	0	17.3%
Qwen 3.5 Plus (2026-02-15)	37	20	14	7	1	15.7%
Gemini 2.5 Pro	29	22	16	12	0	15.6%
GPT-5	25	25	19	5	4	15.6%
Qwen 2.5 72B	27	21	18	9	0	15.1%
DeepSeek-V2 Chat	34	25	8	5	0	14.5%
GPT-4.1 Mini	33	24	16	0	0	14.4%
Qwen 3.5 35B	25	24	20	0	0	13.8%
Gemini 2.5 Flash Lite	27	16	14	7	4	13.7%
Z.AI GLM 4.6	24	23	17	4	0	13.5%
Qwen 3.5 27B	41	27	0	0	0	13.5%
Z.AI GLM 4.7 Flash	29	14	13	12	0	13.4%
GPT-4o, Aug. 6th (temp=0)	32	20	14	0	0	13.3%
Stealth: Healer Alpha	23	19	16	9	0	13.2%
Ministral 3 3B	38	22	3	2	0	13.0%
GPT-4o, May 13th (temp=1)	28	25	9	0	0	12.4%
Claude 3.5 Sonnet	47	5	5	0	0	11.6%
Gemma 3 4B	23	23	6	6	0	11.4%
Claude 3.7 Sonnet	29	16	12	0	0	11.2%
DeepSeek V3.2	30	14	3	0	0	9.3%
GPT-4.1 Nano	24	9	6	5	0	8.8%
Gemini 2.5 Flash	23	14	3	2	0	8.6%
Gemini 3.1 Flash Lite (Preview)	17	11	8	6	0	8.5%
Nemotron 3 Super	18	11	7	7	0	8.4%
Qwen 3.5 Flash	31	4	3	0	0	7.6%
GPT-5 Nano	14	10	8	3	3	7.5%
GPT-4o Mini (temp=0)	17	14	0	0	0	6.1%
Aion 2.0	13	9	8	0	0	6.1%
Inception Mercury 2	30	0	0	0	0	6.0%
Qwen 3.5 122B	17	7	0	0	0	4.9%
Gemini 3 Pro (Preview)	16	7	0	0	0	4.6%
DeepSeek V3.1	10	9	3	0	0	4.5%
Gemma 3 27B	14	3	2	0	0	3.7%
Z.AI GLM 4.7	17	0	0	0	0	3.5%
ByteDance Seed 2.0 Lite	7	5	0	0	0	2.5%
Gemini 3 Flash (Preview)	2	1	0	0	0	0.7%
Gemini 3 Flash (Preview, Reasoning)	3	0	0	0	0	0.6%
Arcee AI: Trinity Mini	1	0	0	0	0	0.2%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4	100	100	100	90	75	92.9%
Claude Sonnet 4.5	100	100	96	95	70	92.4%
Grok 4 Fast	100	98	98	88	78	92.3%
Writer: Palmyra X5	100	100	100	100	58	91.5%
GPT-5.4 (Reasoning)	100	100	100	93	58	90.2%
GPT-5.4 (Reasoning, Low)	100	100	91	86	71	89.7%
Mistral Small 4 (Reasoning)	100	89	89	75	75	85.7%
Mistral Small 4	99	89	82	76	62	81.7%
Grok 4	100	100	74	69	57	80.1%
Ministral 3 14B	100	100	84	63	40	77.4%
GPT-5.4 Mini	97	79	75	64	53	73.5%
Qwen3 235B A22B Instruct 2507	100	98	74	59	35	73.1%
Grok 4.1 Fast	100	75	71	62	52	72.0%
Qwen 3.5 397B A17B	100	83	68	66	31	69.9%
GPT-5.1	96	70	58	58	54	67.3%
Claude Opus 4	91	75	66	65	38	67.0%
Claude Sonnet 4.6	100	83	71	47	31	66.5%
Gemini 3.1 Flash Lite (Preview)	82	72	71	57	50	66.2%
Gemini 2.5 Flash (Reasoning)	88	84	72	44	41	65.8%
MiniMax M2.5	100	65	60	55	47	65.4%
GPT-5.4 Mini (Reasoning, Low)	70	68	67	62	58	65.1%
MoonshotAI: Kimi K2.5	86	65	63	55	52	64.3%
Claude 3.5 Sonnet	97	85	56	49	33	64.0%
Llama 3.1 8B	100	100	100	16	4	63.9%
Rocinante 12B	100	96	88	25	8	63.6%
GPT-5.4 Mini (Reasoning)	92	84	61	50	30	63.4%
Z.AI GLM 5 Turbo	92	68	64	50	40	62.7%
Hermes 3 70B	100	100	76	20	16	62.3%
Gemini 2.5 Flash Lite (Reasoning)	100	70	49	46	32	59.4%
Grok 4.20 (Beta, Reasoning)	78	64	57	56	33	57.6%
Claude Opus 4.6 (Reasoning)	72	68	59	51	36	57.3%
Llama 3.1 Nemotron 70B	96	82	43	41	21	56.7%
Claude Opus 4.5	100	57	44	42	31	54.9%
GPT-5	71	60	57	53	34	54.8%
Mistral NeMO	92	59	49	40	33	54.5%
Gemma 3 12B	96	70	61	29	9	53.3%
Stealth: Hunter Alpha	83	54	48	46	30	52.1%
GPT-4.1 Nano	83	56	51	46	23	51.8%
Mistral Large 2	94	75	60	22	8	51.7%
Qwen 3 32B	100	80	36	25	17	51.6%
Claude Sonnet 4	73	69	54	43	17	51.4%
Claude Sonnet 4.6 (Reasoning)	100	89	33	18	14	50.7%
Stealth: Healer Alpha	76	55	52	38	31	50.6%
Mistral Large 3	92	53	48	45	13	50.2%
Claude Opus 4.6	86	62	47	43	8	49.3%
Mistral Large	64	62	57	53	8	49.0%
GPT-4.1	76	72	55	20	19	48.3%
Gemini 3 Flash (Preview)	80	69	44	27	18	47.6%
GPT-5.4 Nano	63	51	49	48	26	47.4%
Mistral Small Creative	71	57	55	53	0	47.1%
Mistral Medium 3.1	77	47	47	34	31	47.0%
GPT-5.4 Nano (Reasoning, Low)	65	59	56	26	25	46.2%
Hermes 3 405B	100	55	27	24	23	45.6%
Gemini 3.1 Pro (Preview)	100	75	40	12	0	45.3%
DeepSeek V3 (2025-03-24)	79	70	62	14	0	45.1%
GPT-5.4 Nano (Reasoning)	49	48	43	42	40	44.5%
MiniMax M2.7	94	42	31	26	23	43.2%
Claude 3.7 Sonnet	81	50	49	22	11	42.7%
GPT-4o, Aug. 6th (temp=1)	56	55	37	34	30	42.5%
Z.AI GLM 5	55	47	45	32	32	42.3%
DeepSeek V3.2	80	52	40	37	0	41.8%
Arcee AI: Trinity Large (Preview)	62	46	36	31	30	41.1%
Claude Haiku 4.5	60	54	52	22	14	40.2%
GPT-5.2	62	54	38	28	18	40.0%
Ministral 8B	63	48	37	34	15	39.4%
Gemma 3 27B	63	58	32	24	16	38.7%
Claude 3.5 Haiku	100	35	28	25	0	37.5%
Mistral Small 3.2 24B	71	69	43	4	0	37.4%
Grok 4.20 (Beta)	52	48	47	19	17	36.8%
Ministral 3 8B	77	69	38	0	0	36.7%
GPT-4o Mini (temp=1)	69	49	46	18	1	36.7%
WizardLM 2 8x22b	65	48	38	16	11	35.4%
Z.AI GLM 4.6	59	34	29	27	26	35.1%
ByteDance Seed 2.0 Mini	52	51	46	25	0	34.7%
Claude 3 Haiku	71	46	40	16	0	34.6%
ByteDance Seed 1.6	87	49	20	11	0	33.5%
DeepSeek-V2 Chat	75	43	33	11	0	32.6%
ByteDance Seed 1.6 Flash	57	38	38	21	8	32.3%
Qwen 3.5 122B	53	48	43	12	0	31.3%
Gemini 2.5 Pro	67	31	23	20	14	31.0%
Ministral 3B	53	43	29	27	0	30.4%
Gemini 2.5 Flash Lite	55	48	24	20	2	29.8%
o4 Mini High	39	39	32	23	15	29.4%
Gemini 2.5 Flash	64	32	26	19	5	29.1%
Qwen 3.5 Flash	46	46	27	25	0	28.8%
o4 Mini	40	36	35	28	6	28.8%
ByteDance Seed 2.0 Lite	87	41	10	0	0	27.7%
Aion 2.0	53	43	32	9	0	27.7%
GPT-5 Mini	45	35	23	20	14	27.5%
Qwen 3.5 Plus (2026-02-15)	47	35	27	18	7	26.7%
Ministral 3 3B	48	39	27	12	0	25.2%
Z.AI GLM 4.7	52	39	23	12	0	25.1%
Gemini 3 Flash (Preview, Reasoning)	80	26	17	0	0	24.6%
Llama 3.1 70B	51	30	28	7	5	24.2%
Nemotron 3 Nano	79	20	15	2	0	23.1%
GPT-4.1 Mini	36	35	31	11	0	22.6%
LFM2 24B	62	26	20	5	0	22.5%
Cohere Command R+ (Aug. 2024)	57	24	18	10	0	21.9%
GPT-4o, Aug. 6th (temp=0)	49	38	14	5	4	21.8%
Nemotron 3 Super	45	29	16	12	5	21.3%
DeepSeek V3.1	54	35	13	4	0	21.1%
Qwen 3.5 35B	71	18	9	5	0	20.7%
Z.AI GLM 4.5	56	21	12	11	0	19.9%
GPT-4o, May 13th (temp=1)	31	21	20	14	1	17.2%
GPT-4o, May 13th (temp=0)	43	25	16	0	0	17.0%
Qwen 3.5 27B	40	22	14	5	0	16.2%
Gemini 3 Pro (Preview)	26	25	13	9	5	15.5%
Gemma 3 4B	30	22	19	7	0	15.5%
DeepSeek V3 (2024-12-26)	46	11	10	2	0	13.7%
Qwen 2.5 72B	31	17	13	7	0	13.6%
Z.AI GLM 4.7 Flash	22	17	13	11	5	13.5%
Qwen 3.5 9B	26	26	3	0	0	11.0%
GPT-5 Nano	23	10	9	8	0	9.8%
GPT-4o Mini (temp=0)	21	14	0	0	0	7.0%
Arcee AI: Trinity Mini	34	0	0	0	0	6.7%
Inception Mercury	9	0	0	0	0	1.9%
Inception Mercury 2	7	0	0	0	0	1.4%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Rocinante 12B	100	100	100	87	69	91.3%
GPT-5.4	100	92	84	80	79	87.0%
Llama 3.1 8B	100	100	100	93	9	80.4%
GPT-5.4 Mini (Reasoning, Low)	100	86	84	73	54	79.5%
GPT-5.4 (Reasoning)	92	83	82	75	56	77.5%
GPT-5.4 Mini (Reasoning)	93	93	80	72	42	75.9%
Qwen3 235B A22B Instruct 2507	86	85	77	70	46	72.7%
GPT-5.4 Mini	92	88	72	52	50	70.9%
Claude 3.5 Haiku	100	100	75	51	19	69.0%
Mistral Small 4	100	95	54	49	46	68.8%
GPT-5.4 (Reasoning, Low)	76	70	66	60	53	64.7%
Ministral 3 14B	88	77	65	52	38	64.0%
MiniMax M2.7	79	68	62	55	49	62.8%
Mistral Small 4 (Reasoning)	92	63	58	43	42	59.9%
Llama 3.1 70B	100	80	49	47	13	57.9%
Grok 4	87	79	42	40	38	57.3%
Mistral Large 3	100	72	45	36	31	56.8%
Writer: Palmyra X5	100	87	60	32	0	55.9%
GPT-4.1	71	65	60	49	33	55.5%
Grok 4.1 Fast	81	60	50	50	36	55.3%
Claude Sonnet 4.5	85	72	68	39	10	54.9%
Grok 4 Fast	75	69	52	41	34	54.4%
Claude Opus 4.5	80	61	53	42	32	53.8%
Hermes 3 405B	100	52	48	45	18	52.7%
Claude Opus 4.6	90	49	49	43	28	51.9%
GPT-5.1	69	53	49	42	38	50.3%
Claude Opus 4	92	59	43	27	19	48.1%
ByteDance Seed 1.6 Flash	63	57	56	42	20	47.5%
DeepSeek V3 (2025-03-24)	100	68	34	34	0	47.3%
Gemini 3.1 Pro (Preview)	90	81	33	27	0	46.2%
Grok 4.20 (Beta)	72	62	46	26	25	46.1%
Qwen 3.5 397B A17B	68	59	56	30	10	44.5%
Claude Haiku 4.5	77	60	37	34	14	44.4%
Claude 3 Haiku	64	58	56	22	20	44.0%
Z.AI GLM 5	77	52	44	31	15	44.0%
Grok 4.20 (Beta, Reasoning)	61	55	38	36	30	43.8%
Cohere Command R+ (Aug. 2024)	66	43	39	37	33	43.6%
MoonshotAI: Kimi K2.5	60	56	51	36	0	40.5%
Qwen 3.5 35B	72	43	41	24	19	40.0%
Mistral Small Creative	74	45	42	38	0	39.7%
Claude Opus 4.6 (Reasoning)	58	52	37	29	22	39.5%
Z.AI GLM 5 Turbo	54	53	42	35	14	39.4%
GPT-5.4 Nano (Reasoning)	62	47	42	30	14	39.2%
Ministral 3 8B	69	56	43	27	0	39.2%
MiniMax M2.5	90	42	34	27	0	38.7%
Ministral 8B	66	59	51	10	4	37.8%
Gemini 2.5 Pro	58	46	34	24	24	37.2%
GPT-5 Mini	50	40	31	30	27	35.7%
GPT-4o Mini (temp=1)	58	36	33	30	19	35.5%
Claude Sonnet 4	63	55	28	24	0	34.3%
Mistral Medium 3.1	43	41	39	27	20	34.0%
GPT-5.4 Nano (Reasoning, Low)	46	39	37	36	11	33.9%
Qwen 3 32B	56	52	37	21	0	33.2%
Arcee AI: Trinity Large (Preview)	56	46	30	21	7	31.9%
Gemini 2.5 Flash (Reasoning)	65	31	22	21	17	31.5%
DeepSeek-V2 Chat	78	27	24	20	5	30.8%
DeepSeek V3 (2024-12-26)	51	30	30	26	13	30.1%
GPT-4o, May 13th (temp=0)	61	29	28	20	10	29.8%
Mistral Large	50	47	46	6	0	29.6%
Gemini 2.5 Flash Lite	56	39	38	11	0	29.0%
Hermes 3 70B	72	33	27	11	0	28.8%
Mistral Large 2	69	36	26	11	1	28.7%
Mistral Small 3.2 24B	49	41	21	18	14	28.3%
Z.AI GLM 4.5	48	32	26	24	9	28.0%
Claude 3.5 Sonnet	31	30	28	25	24	27.6%
Gemini 3.1 Flash Lite (Preview)	49	44	44	0	0	27.5%
LFM2 24B	42	38	26	17	13	27.4%
GPT-5.4 Nano	35	34	26	23	19	27.4%
o4 Mini	49	47	27	9	3	27.1%
GPT-4.1 Nano	75	35	19	0	0	26.0%
Gemini 2.5 Flash Lite (Reasoning)	37	34	26	18	14	26.0%
Llama 3.1 Nemotron 70B	51	29	19	17	11	25.5%
Stealth: Hunter Alpha	37	34	22	17	15	25.0%
Aion 2.0	51	41	19	14	0	24.9%
DeepSeek V3.1	61	28	25	8	0	24.4%
o4 Mini High	54	34	27	0	0	23.0%
Ministral 3B	40	38	28	9	0	22.9%
GPT-4.1 Mini	45	36	17	16	1	22.8%
GPT-5.2	35	29	19	16	13	22.4%
DeepSeek V3.2	47	40	17	4	3	22.2%
Gemma 3 27B	48	31	21	11	0	22.0%
GPT-4o, May 13th (temp=1)	56	36	17	0	0	21.8%
Ministral 3 3B	37	33	22	8	0	19.9%
GPT-4o, Aug. 6th (temp=1)	34	30	25	9	0	19.6%
GPT-5	36	22	18	11	10	19.3%
Mistral NeMO	55	20	19	0	0	18.8%
Gemini 2.5 Flash	45	23	15	7	3	18.5%
Z.AI GLM 4.6	31	28	25	7	0	18.3%
Claude Sonnet 4.6 (Reasoning)	38	28	10	9	6	18.2%
Claude 3.7 Sonnet	38	27	20	1	0	17.0%
Stealth: Healer Alpha	45	30	9	0	0	16.8%
WizardLM 2 8x22b	58	12	8	2	0	16.1%
ByteDance Seed 1.6	37	18	13	9	3	15.9%
Qwen 3.5 Flash	44	21	9	0	0	14.8%
GPT-4o, Aug. 6th (temp=0)	38	27	6	0	0	14.3%
ByteDance Seed 2.0 Lite	30	16	12	5	2	12.8%
ByteDance Seed 2.0 Mini	27	25	10	2	0	12.7%
GPT-4o Mini (temp=0)	43	20	0	0	0	12.7%
Stealth: Aurora Alpha	60	0	0	0	0	12.0%
Gemma 3 4B	31	18	6	3	0	11.8%
Gemini 3 Flash (Preview)	28	15	10	0	0	10.5%
Qwen 2.5 72B	31	13	8	0	0	10.5%
Qwen 3.5 27B	42	8	0	0	0	9.9%
Nemotron 3 Super	30	16	0	0	0	9.2%
Z.AI GLM 4.7	31	7	3	0	0	8.3%
Inception Mercury	41	0	0	0	0	8.2%
Claude Sonnet 4.6	31	5	4	0	0	8.1%
Gemma 3 12B	23	12	5	0	0	8.0%
Gemini 3 Flash (Preview, Reasoning)	17	12	7	3	0	7.7%
Gemini 3 Pro (Preview)	14	10	9	6	0	7.7%
Qwen 3.5 122B	20	8	4	3	0	7.0%
Nemotron 3 Nano	14	6	4	0	0	4.9%
Z.AI GLM 4.7 Flash	10	8	7	0	0	4.8%
Qwen 3.5 9B	19	0	0	0	0	3.9%
Arcee AI: Trinity Mini	19	0	0	0	0	3.8%
GPT-5 Nano	10	3	2	0	0	3.0%
Inception Mercury 2	12	0	0	0	0	2.5%
Qwen 3.5 Plus (2026-02-15)	10	1	0	0	0	2.2%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Llama 3.1 8B	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	93	85	95.6%
GPT-5.4	100	98	96	93	82	93.8%
Writer: Palmyra X5	100	100	100	100	64	92.6%
Qwen3 235B A22B Instruct 2507	100	100	100	80	78	91.7%
Mistral Small 4 (Reasoning)	100	100	80	78	75	86.5%
GPT-5.4 (Reasoning, Low)	100	87	75	71	66	79.8%
Claude Sonnet 4.5	100	93	69	67	66	79.1%
GPT-5.1	100	87	81	62	62	78.4%
Claude 3 Haiku	95	88	70	69	67	77.9%
Hermes 3 70B	100	100	90	63	17	74.2%
Claude 3.5 Haiku	100	85	68	58	57	73.6%
GPT-4o Mini (temp=1)	100	79	71	66	52	73.5%
Z.AI GLM 4.5	85	79	79	66	57	72.9%
Z.AI GLM 5	100	91	63	62	45	72.2%
Claude Opus 4.5	100	90	68	63	39	72.1%
Gemini 2.5 Flash Lite	100	82	76	66	35	71.9%
Llama 3.1 70B	90	82	77	56	51	71.2%
Claude Sonnet 4	95	93	81	43	40	70.7%
GPT-5.4 Mini (Reasoning)	100	84	67	67	35	70.7%
MoonshotAI: Kimi K2.5	87	79	70	63	51	70.1%
Claude Haiku 4.5	100	71	62	60	52	68.9%
GPT-4.1 Mini	100	87	56	52	44	67.9%
MiniMax M2.5	100	77	65	51	46	67.7%
Z.AI GLM 5 Turbo	100	80	68	52	37	67.5%
MiniMax M2.7	89	89	66	52	33	65.7%
Claude Sonnet 4.6 (Reasoning)	87	68	60	53	53	64.1%
Rocinante 12B	100	62	57	52	48	63.6%
Mistral Small 4	100	96	51	42	29	63.5%
DeepSeek V3 (2025-03-24)	99	67	66	46	34	62.4%
GPT-5.4 Mini	81	78	75	41	36	62.4%
GPT-4.1	99	63	57	51	41	62.1%
Gemini 2.5 Flash Lite (Reasoning)	79	74	61	59	36	61.8%
GPT-5.4 Mini (Reasoning, Low)	78	68	57	54	50	61.3%
Claude Opus 4	100	69	55	43	36	60.6%
Stealth: Hunter Alpha	83	64	59	49	47	60.6%
Grok 4.20 (Beta, Reasoning)	73	67	61	51	47	59.9%
Hermes 3 405B	100	88	53	47	8	59.2%
Ministral 3 8B	88	74	48	45	40	58.9%
GPT-4.1 Nano	99	83	73	26	11	58.4%
o4 Mini High	79	77	48	44	41	57.8%
Qwen 3.5 397B A17B	84	68	55	49	33	57.7%
GPT-4o, Aug. 6th (temp=1)	92	85	54	40	18	57.7%
Llama 3.1 Nemotron 70B	78	62	58	48	39	56.9%
Gemini 3.1 Pro (Preview)	85	60	51	46	40	56.4%
Grok 4 Fast	81	78	57	43	17	55.2%
Claude 3.5 Sonnet	91	65	53	36	23	53.6%
Grok 4	85	59	49	36	34	52.5%
Gemini 2.5 Flash (Reasoning)	74	70	44	40	34	52.3%
GPT-4o, Aug. 6th (temp=0)	92	77	41	32	16	51.7%
Claude Opus 4.6 (Reasoning)	75	58	43	41	38	51.0%
Ministral 8B	100	70	58	16	11	51.0%
o4 Mini	69	65	51	36	31	50.6%
GPT-5.4 Nano	69	53	51	49	30	50.5%
Gemma 3 12B	75	53	48	40	36	50.3%
LFM2 24B	77	63	49	36	20	49.2%
Grok 4.1 Fast	65	53	48	41	38	49.1%
GPT-5.4 Nano (Reasoning)	58	56	54	41	36	49.1%
Claude Sonnet 4.6	96	52	40	30	27	48.9%
GPT-5.2	61	59	45	43	36	48.8%
Claude 3.7 Sonnet	70	68	49	33	21	48.2%
GPT-5.4 Nano (Reasoning, Low)	62	56	51	43	26	47.6%
Mistral Medium 3.1	63	57	43	41	32	47.1%
Grok 4.20 (Beta)	60	56	50	39	29	46.9%
Claude Opus 4.6	61	46	46	43	25	44.0%
Ministral 3 3B	90	50	45	22	7	42.9%
Z.AI GLM 4.6	71	54	37	31	16	41.8%
Gemini 2.5 Pro	51	48	48	36	26	41.8%
Z.AI GLM 4.7 Flash	64	56	45	31	12	41.7%
Mistral NeMO	84	62	38	12	12	41.6%
Qwen 3.5 Plus (2026-02-15)	53	51	46	38	20	41.6%
GPT-4o, May 13th (temp=1)	65	49	40	34	12	39.9%
Ministral 3 14B	80	42	36	29	11	39.7%
Mistral Large 3	61	49	46	29	12	39.5%
Gemini 2.5 Flash	71	63	36	19	8	39.2%
Mistral Small Creative	64	48	43	34	6	39.0%
Qwen 3.5 9B	67	58	46	23	0	38.8%
Gemma 3 27B	66	47	39	35	8	38.8%
DeepSeek V3.2	59	44	36	32	20	38.1%
Cohere Command R+ (Aug. 2024)	77	33	32	28	19	38.0%
DeepSeek V3.1	54	54	36	35	10	37.8%
Aion 2.0	55	53	42	39	0	37.8%
Arcee AI: Trinity Large (Preview)	55	50	45	37	0	37.5%
DeepSeek-V2 Chat	76	34	34	23	12	35.8%
ByteDance Seed 1.6 Flash	54	36	35	27	25	35.4%
Stealth: Healer Alpha	52	42	37	31	11	34.5%
Qwen 3.5 35B	86	44	17	13	8	33.6%
Qwen 3 32B	51	49	43	23	0	33.2%
Mistral Large	51	43	37	27	3	32.3%
ByteDance Seed 2.0 Lite	100	39	20	2	0	32.0%
Nemotron 3 Super	54	37	36	30	0	31.5%
WizardLM 2 8x22b	100	53	4	0	0	31.4%
GPT-5	56	33	30	25	9	30.6%
GPT-4o, May 13th (temp=0)	70	54	25	3	0	30.5%
Mistral Large 2	43	42	34	31	0	30.0%
DeepSeek V3 (2024-12-26)	96	19	17	17	0	29.8%
Qwen 3.5 Flash	49	37	35	14	8	28.5%
GPT-5 Mini	53	39	26	16	9	28.4%
Ministral 3B	68	20	18	17	2	25.0%
Mistral Small 3.2 24B	72	21	19	0	0	22.5%
Gemma 3 4B	48	29	18	14	0	21.8%
Gemini 3 Flash (Preview)	33	24	19	17	13	21.3%
Arcee AI: Trinity Mini	40	31	20	9	5	21.1%
Gemini 3 Flash (Preview, Reasoning)	43	18	17	16	10	20.7%
Qwen 3.5 27B	34	27	18	17	7	20.6%
Nemotron 3 Nano	31	27	20	15	5	19.6%
Gemini 3.1 Flash Lite (Preview)	30	27	22	15	0	18.6%
GPT-4o Mini (temp=0)	25	23	14	13	11	17.2%
ByteDance Seed 2.0 Mini	33	21	16	9	6	16.7%
Gemini 3 Pro (Preview)	32	24	18	4	0	15.5%
Qwen 2.5 72B	49	16	11	0	0	15.4%
ByteDance Seed 1.6	45	12	10	6	0	14.5%
Qwen 3.5 122B	37	15	7	6	0	12.9%
GPT-5 Nano	23	15	12	8	0	11.7%
Z.AI GLM 4.7	28	14	12	2	0	11.2%
Inception Mercury	6	0	0	0	0	1.3%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Hermes 3 70B	100	100	94	81	42	83.4%
Qwen3 235B A22B Instruct 2507	84	82	77	75	70	77.4%
Rocinante 12B	100	93	73	60	60	77.2%
GPT-5.4	95	86	84	62	54	76.4%
Writer: Palmyra X5	100	83	78	65	43	74.0%
Llama 3.1 8B	100	100	74	56	23	70.6%
GPT-5.4 (Reasoning)	85	80	71	66	49	70.1%
Claude Sonnet 4.6	100	79	77	46	23	64.9%
GPT-5.4 (Reasoning, Low)	82	82	76	43	32	63.0%
Claude Opus 4.6 (Reasoning)	84	73	58	49	48	62.5%
Llama 3.1 Nemotron 70B	100	100	73	27	13	62.4%
Claude Opus 4.6	76	67	66	46	41	59.2%
ByteDance Seed 2.0 Lite	85	85	79	21	12	56.3%
Hermes 3 405B	98	82	63	23	16	56.1%
WizardLM 2 8x22b	80	62	57	55	21	55.2%
Mistral Small Creative	94	70	48	33	29	55.0%
GPT-5.1	72	55	49	43	41	52.3%
Claude 3 Haiku	73	64	43	40	35	51.2%
Claude Sonnet 4.5	69	49	46	45	42	50.2%
Gemma 3 12B	100	51	43	39	16	50.0%
Claude Sonnet 4.6 (Reasoning)	77	63	50	31	24	48.9%
Gemma 3 4B	63	60	53	47	20	48.6%
Llama 3.1 70B	100	43	41	37	21	48.4%
Mistral Small 4	60	59	51	32	23	45.0%
GPT-5.4 Mini	63	46	42	39	33	44.4%
Claude Haiku 4.5	78	57	43	27	16	44.3%
Z.AI GLM 5	85	50	36	25	25	44.1%
Claude Opus 4.5	82	55	34	26	21	43.5%
Gemini 3.1 Flash Lite (Preview)	60	56	45	27	26	42.8%
GPT-5.4 Mini (Reasoning, Low)	71	41	39	35	23	41.9%
Claude Opus 4	67	64	42	34	0	41.6%
ByteDance Seed 1.6 Flash	67	58	43	25	12	40.8%
Claude Sonnet 4	72	42	39	33	18	40.7%
Z.AI GLM 4.5	66	50	36	27	17	39.4%
Grok 4.1 Fast	75	53	25	21	20	39.0%
MiniMax M2.5	100	48	39	5	3	39.0%
Gemini 2.5 Flash	58	51	50	26	4	37.7%
DeepSeek V3.1	63	51	37	21	15	37.4%
Ministral 3 14B	60	43	36	25	20	36.9%
Claude 3.7 Sonnet	70	50	34	20	10	36.7%
GPT-4o, Aug. 6th (temp=1)	65	62	21	21	10	35.8%
Grok 4.20 (Beta, Reasoning)	45	42	35	33	23	35.4%
Mistral Small 4 (Reasoning)	58	48	37	20	11	34.7%
DeepSeek V3 (2025-03-24)	71	71	14	5	3	32.8%
Cohere Command R+ (Aug. 2024)	100	41	21	0	0	32.5%
Gemini 2.5 Flash (Reasoning)	53	39	34	21	14	32.4%
GPT-5.4 Mini (Reasoning)	55	49	30	14	13	32.1%
Stealth: Healer Alpha	50	47	41	17	2	31.3%
Gemini 3 Flash (Preview, Reasoning)	49	39	31	23	12	30.9%
Gemini 2.5 Pro	70	40	32	2	0	29.0%
Gemini 2.5 Flash Lite	46	44	31	21	2	28.9%
GPT-4o, Aug. 6th (temp=0)	74	34	25	7	3	28.5%
DeepSeek V3.2	37	33	27	25	20	28.3%
Grok 4	51	33	27	23	7	28.2%
ByteDance Seed 2.0 Mini	49	39	33	19	0	28.0%
Stealth: Hunter Alpha	46	34	33	21	5	27.8%
GPT-4o, May 13th (temp=1)	39	33	26	25	15	27.3%
GPT-4o Mini (temp=1)	47	45	21	13	10	27.0%
Gemini 2.5 Flash Lite (Reasoning)	47	31	29	26	0	26.7%
GPT-4.1	59	36	19	19	0	26.6%
Ministral 3 8B	81	34	16	0	0	26.3%
Grok 4.20 (Beta)	40	34	25	16	16	26.3%
Mistral Medium 3.1	46	29	28	16	12	26.1%
Z.AI GLM 4.6	45	25	23	20	17	26.1%
MoonshotAI: Kimi K2.5	43	37	29	16	0	25.0%
Mistral Large 2	48	38	25	13	0	24.6%
o4 Mini	40	38	23	21	0	24.4%
GPT-5.4 Nano	34	29	25	18	14	24.2%
Qwen 3.5 397B A17B	43	31	27	12	7	24.1%
o4 Mini High	64	22	17	15	3	24.1%
GPT-4o, May 13th (temp=0)	41	35	26	15	0	23.3%
Grok 4 Fast	48	27	25	9	6	22.9%
GPT-5.2	45	21	20	19	5	22.2%
Z.AI GLM 5 Turbo	40	34	23	11	0	21.6%
Arcee AI: Trinity Large (Preview)	42	29	20	9	8	21.4%
Gemini 3 Flash (Preview)	51	29	12	9	3	20.7%
Claude 3.5 Sonnet	38	38	26	0	0	20.4%
Claude 3.5 Haiku	37	31	20	10	0	19.7%
MiniMax M2.7	42	27	13	8	3	18.7%
GPT-5.4 Nano (Reasoning)	31	24	24	14	0	18.5%
Qwen 3.5 35B	57	35	0	0	0	18.5%
Mistral NeMO	50	30	11	0	0	18.1%
GPT-4.1 Mini	45	28	9	7	2	18.1%
Mistral Large 3	45	23	16	5	2	18.1%
Qwen 3 32B	43	28	19	0	0	17.9%
Gemini 3 Pro (Preview)	34	26	14	7	4	17.1%
GPT-5	31	23	17	14	0	17.0%
Mistral Large	38	27	19	0	0	16.6%
Gemma 3 27B	56	21	3	0	0	16.1%
GPT-4.1 Nano	23	17	16	16	7	15.7%
Ministral 8B	36	23	16	0	0	14.9%
LFM2 24B	39	20	11	0	0	13.9%
Z.AI GLM 4.7	29	23	12	0	0	12.9%
GPT-5 Mini	37	18	7	0	0	12.5%
DeepSeek V3 (2024-12-26)	27	14	13	8	0	12.3%
GPT-5.4 Nano (Reasoning, Low)	21	16	14	7	2	11.9%
Aion 2.0	40	10	3	2	0	10.8%
Qwen 2.5 72B	24	15	9	0	0	9.5%
Ministral 3B	27	18	1	0	0	9.0%
Mistral Small 3.2 24B	24	13	6	0	0	8.7%
Qwen 3.5 Plus (2026-02-15)	42	1	0	0	0	8.6%
Qwen 3.5 Flash	16	11	7	0	0	6.7%
Z.AI GLM 4.7 Flash	24	4	0	0	0	5.5%
DeepSeek-V2 Chat	9	6	0	0	0	3.0%
Qwen 3.5 9B	13	2	0	0	0	3.0%
Gemini 3.1 Pro (Preview)	10	2	1	0	0	2.7%
GPT-4o Mini (temp=0)	6	5	0	0	0	2.2%
Nemotron 3 Nano	7	0	0	0	0	1.5%
Qwen 3.5 27B	7	0	0	0	0	1.4%
Nemotron 3 Super	7	0	0	0	0	1.3%
Ministral 3 3B	3	2	0	0	0	1.0%
Qwen 3.5 122B	3	1	0	0	0	0.8%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	97	99.4%
Writer: Palmyra X5	100	100	100	100	91	98.2%
Claude Sonnet 4.5	100	100	100	99	86	97.0%
Cohere Command R+ (Aug. 2024)	100	100	95	91	85	94.2%
Claude Opus 4.5	99	92	91	84	84	89.9%
GPT-4o, Aug. 6th (temp=1)	100	94	89	87	78	89.4%
Claude 3 Haiku	100	100	97	87	55	87.8%
Claude Opus 4.6 (Reasoning)	100	97	93	77	69	87.0%
Claude Sonnet 4.6 (Reasoning)	100	100	85	80	69	86.8%
Gemma 3 27B	99	91	89	86	65	86.2%
Grok 4.1 Fast	100	100	89	84	57	86.2%
Llama 3.1 8B	100	100	100	79	51	86.0%
GPT-5.4	100	89	87	83	66	85.1%
Claude Sonnet 4	100	90	83	83	64	84.0%
GPT-5.4 (Reasoning)	100	88	86	85	60	83.9%
Hermes 3 70B	100	100	100	69	47	83.2%
Mistral Small 4 (Reasoning)	100	100	100	79	34	82.5%
Claude Opus 4.6	100	100	74	72	64	82.0%
Z.AI GLM 5 Turbo	100	93	80	72	64	81.8%
MiniMax M2.5	96	93	81	68	57	79.2%
GPT-5.4 (Reasoning, Low)	100	83	77	74	55	77.8%
Hermes 3 405B	100	97	71	59	56	76.6%
Claude Opus 4	100	100	92	57	31	76.0%
MiniMax M2.7	100	72	68	68	61	73.8%
Stealth: Hunter Alpha	100	76	75	73	44	73.6%
Ministral 8B	100	99	81	67	3	69.9%
Z.AI GLM 5	88	82	66	63	44	68.6%
GPT-5.1	100	81	65	54	39	67.7%
Claude 3.5 Haiku	96	92	79	47	23	67.3%
Claude Haiku 4.5	100	98	57	42	34	66.1%
Mistral Small 4	100	84	60	54	19	63.5%
Stealth: Healer Alpha	89	71	71	39	36	61.3%
GPT-5.4 Mini	81	71	55	54	45	61.3%
Claude Sonnet 4.6	92	71	70	61	0	59.1%
Mistral Large 2	83	70	65	47	22	57.3%
GPT-5.4 Mini (Reasoning, Low)	69	63	57	51	46	57.2%
Llama 3.1 Nemotron 70B	79	70	58	45	31	56.6%
ByteDance Seed 1.6 Flash	73	65	63	55	27	56.5%
Grok 4	88	64	47	44	38	56.2%
Arcee AI: Trinity Large (Preview)	75	60	52	47	46	56.0%
DeepSeek V3 (2025-03-24)	89	63	54	40	30	55.1%
Ministral 3 14B	84	70	66	41	7	53.5%
Mistral Small Creative	87	66	46	35	31	53.0%
WizardLM 2 8x22b	81	76	44	34	29	52.9%
GPT-4.1 Nano	61	57	52	51	34	51.2%
Grok 4 Fast	69	66	48	41	31	51.0%
ByteDance Seed 2.0 Mini	70	60	58	35	32	50.9%
Z.AI GLM 4.6	84	64	48	31	23	50.1%
Gemma 3 12B	65	53	49	42	38	49.5%
Grok 4.20 (Beta)	67	61	50	41	27	49.1%
Aion 2.0	56	50	50	47	40	48.6%
Mistral Large 3	79	64	49	23	23	47.6%
GPT-4.1	50	49	49	48	40	47.4%
Ministral 3 8B	78	52	48	40	12	45.9%
Mistral Large	96	51	41	33	7	45.6%
Gemini 2.5 Flash Lite (Reasoning)	62	61	51	34	13	44.4%
Gemini 2.5 Flash Lite	56	51	45	36	33	44.2%
DeepSeek V3.2	54	45	41	40	38	43.7%
GPT-4.1 Mini	77	46	39	38	17	43.6%
GPT-5.4 Nano	53	51	47	34	33	43.4%
Mistral Medium 3.1	59	52	51	43	11	43.4%
GPT-4o, May 13th (temp=1)	51	50	42	36	36	43.0%
GPT-5.4 Nano (Reasoning)	54	50	49	34	27	42.8%
Claude 3.7 Sonnet	81	63	37	24	8	42.7%
GPT-4o Mini (temp=1)	70	60	31	23	21	40.9%
Qwen 3.5 Plus (2026-02-15)	52	40	36	32	29	37.8%
Gemini 2.5 Pro	78	44	38	18	9	37.5%
GPT-5.4 Mini (Reasoning)	39	36	35	35	34	35.9%
DeepSeek V3 (2024-12-26)	100	53	23	3	0	35.7%
Claude 3.5 Sonnet	60	45	45	24	0	34.6%
DeepSeek-V2 Chat	85	60	28	0	0	34.5%
GPT-5	84	40	35	7	6	34.3%
Llama 3.1 70B	60	57	24	16	11	33.5%
Z.AI GLM 4.7	51	46	29	20	18	32.7%
Ministral 3B	60	51	24	11	10	31.2%
Z.AI GLM 4.7 Flash	59	42	36	8	5	30.3%
Arcee AI: Trinity Mini	55	51	35	8	0	29.8%
Gemini 3 Pro (Preview)	46	31	28	28	11	28.8%
DeepSeek V3.1	50	29	26	24	13	28.5%
Gemini 3.1 Flash Lite (Preview)	71	43	20	0	0	26.8%
GPT-5.2	40	40	29	19	6	26.8%
Gemini 2.5 Flash (Reasoning)	43	35	24	20	12	26.7%
Mistral NeMO	35	27	26	23	21	26.4%
Qwen 3 32B	39	36	28	21	2	25.3%
Gemma 3 4B	45	38	21	18	0	24.5%
o4 Mini High	52	43	17	8	0	24.1%
Mistral Small 3.2 24B	67	33	21	0	0	24.1%
Gemini 2.5 Flash	52	27	21	20	0	24.1%
ByteDance Seed 2.0 Lite	48	33	16	13	5	23.1%
GPT-5.4 Nano (Reasoning, Low)	28	25	21	19	14	21.5%
Qwen 3.5 9B	54	27	26	0	0	21.3%
MoonshotAI: Kimi K2.5	43	30	15	8	7	20.7%
Z.AI GLM 4.5	35	30	26	0	0	18.1%
LFM2 24B	27	26	13	11	9	17.0%
Ministral 3 3B	61	16	8	0	0	16.9%
o4 Mini	39	24	14	7	0	16.7%
GPT-5 Mini	33	28	13	0	0	14.8%
Gemini 3 Flash (Preview)	30	28	5	5	4	14.4%
Grok 4.20 (Beta, Reasoning)	54	10	5	3	0	14.4%
Qwen 3.5 397B A17B	20	20	10	7	0	11.5%
Gemini 3 Flash (Preview, Reasoning)	33	13	6	1	0	10.7%
GPT-4o, Aug. 6th (temp=0)	28	22	0	0	0	9.9%
Qwen 3.5 Flash	25	20	2	0	0	9.5%
Qwen 2.5 72B	40	6	1	0	0	9.5%
Nemotron 3 Super	19	17	8	0	0	8.8%
Qwen 3.5 35B	26	16	0	0	0	8.4%
Gemini 3.1 Pro (Preview)	15	13	2	0	0	5.9%
Qwen 3.5 122B	29	0	0	0	0	5.8%
GPT-5 Nano	21	7	0	0	0	5.6%
GPT-4o, May 13th (temp=0)	12	5	0	0	0	3.3%
GPT-4o Mini (temp=0)	7	5	3	0	0	3.0%
ByteDance Seed 1.6	3	0	0	0	0	0.5%
Nemotron 3 Nano	1	0	0	0	0	0.2%
Qwen 3.5 27B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Llama 3.1 8B	100	100	72	51	40	72.6%
Writer: Palmyra X5	99	82	66	56	18	64.2%
Qwen3 235B A22B Instruct 2507	87	69	62	61	35	62.9%
Llama 3.1 Nemotron 70B	100	100	74	25	11	62.0%
Ministral 3 14B	100	83	58	36	24	60.3%
GPT-5 Nano	100	100	100	0	0	60.1%
Mistral Large 2	98	74	60	51	13	59.2%
Llama 3.1 70B	83	63	62	45	42	59.0%
Grok 4.1 Fast	100	92	59	40	0	58.2%
Grok 4.20 (Beta, Reasoning)	90	68	62	45	22	57.4%
Mistral Small 4	78	68	49	49	28	54.5%
Z.AI GLM 5	100	58	56	35	21	54.0%
Mistral Small 4 (Reasoning)	84	67	48	41	19	51.6%
Mistral Small Creative	85	64	54	38	12	50.6%
Claude Opus 4.5	56	54	47	41	21	43.8%
Mistral NeMO	100	100	16	0	0	43.1%
Rocinante 12B	100	100	11	0	0	42.2%
Cohere Command R+ (Aug. 2024)	85	68	56	0	0	41.9%
Hermes 3 405B	80	68	49	0	0	39.2%
o4 Mini High	79	75	35	4	0	38.4%
Claude Sonnet 4.6	79	58	24	24	6	38.2%
Claude Sonnet 4.5	95	48	24	23	0	38.0%
Ministral 3 8B	94	46	24	18	4	37.5%
Mistral Medium 3.1	58	47	41	37	3	37.1%
Arcee AI: Trinity Large (Preview)	100	67	16	0	0	36.5%
Claude Sonnet 4	66	55	53	5	0	36.0%
Grok 4 Fast	57	46	29	23	21	35.1%
Mistral Large 3	52	43	40	25	14	34.5%
Hermes 3 70B	89	35	23	20	0	33.2%
GPT-5.4	39	34	28	24	18	28.5%
Claude 3.5 Haiku	75	65	0	0	0	28.1%
Grok 4.20 (Beta)	43	41	28	14	2	25.8%
Claude Opus 4	35	33	25	22	11	25.1%
Claude Sonnet 4.6 (Reasoning)	40	27	24	24	5	24.1%
MiniMax M2.7	50	30	25	14	0	23.8%
Claude Haiku 4.5	52	49	9	6	0	23.1%
Qwen 3 32B	71	27	9	0	0	21.4%
Claude 3.7 Sonnet	53	25	21	3	0	20.2%
MiniMax M2.5	70	15	14	1	0	20.0%
Claude Opus 4.6	40	24	19	16	1	19.8%
Mistral Small 3.2 24B	80	19	0	0	0	19.8%
MoonshotAI: Kimi K2.5	51	20	16	4	3	18.7%
Z.AI GLM 5 Turbo	42	31	18	2	0	18.7%
GPT-4o, May 13th (temp=1)	73	16	1	0	0	18.1%
LFM2 24B	40	33	16	1	0	18.0%
Z.AI GLM 4.6	53	23	12	1	0	18.0%
Claude Opus 4.6 (Reasoning)	35	27	24	2	0	17.6%
DeepSeek V3 (2024-12-26)	48	16	13	3	0	15.9%
Qwen 3.5 27B	70	6	0	0	0	15.2%
GPT-4o, Aug. 6th (temp=1)	29	26	19	3	0	15.2%
Gemma 3 27B	55	14	5	0	0	14.8%
o4 Mini	31	16	16	10	0	14.4%
GPT-5.4 (Reasoning)	29	13	13	10	6	14.2%
Stealth: Healer Alpha	54	13	3	0	0	14.1%
GPT-5	36	33	0	0	0	13.9%
GPT-4.1	23	22	22	0	0	13.1%
GPT-5.4 Mini	26	19	10	3	2	11.9%
Z.AI GLM 4.5	48	5	2	0	0	10.9%
Mistral Large	28	19	3	3	0	10.6%
ByteDance Seed 2.0 Lite	49	3	0	0	0	10.6%
DeepSeek V3 (2025-03-24)	35	17	0	0	0	10.5%
GPT-4o, May 13th (temp=0)	41	11	0	0	0	10.3%
Stealth: Hunter Alpha	36	14	0	0	0	10.0%
GPT-5.4 (Reasoning, Low)	18	17	11	0	0	9.0%
DeepSeek-V2 Chat	25	20	0	0	0	9.0%
Ministral 8B	35	10	0	0	0	8.9%
Gemini 3.1 Flash Lite (Preview)	30	11	0	0	0	8.2%
ByteDance Seed 1.6 Flash	25	16	0	0	0	8.1%
DeepSeek V3.1	39	0	0	0	0	7.7%
Grok 4	35	2	0	0	0	7.3%
Ministral 3B	16	14	6	0	0	7.2%
GPT-5.4 Mini (Reasoning, Low)	23	10	0	0	0	6.6%
GPT-4o Mini (temp=1)	20	12	0	0	0	6.4%
Gemini 2.5 Pro	20	8	4	0	0	6.3%
Gemini 3 Pro (Preview)	20	11	0	0	0	6.3%
Qwen 2.5 72B	30	0	0	0	0	6.0%
GPT-5.4 Mini (Reasoning)	15	11	2	0	0	5.7%
Aion 2.0	16	10	0	0	0	5.1%
Qwen 3.5 397B A17B	14	6	5	0	0	5.1%
Claude 3 Haiku	23	3	0	0	0	5.0%
Gemma 3 4B	14	8	0	0	0	4.3%
Qwen 3.5 35B	20	0	0	0	0	4.0%
Z.AI GLM 4.7	19	0	0	0	0	3.8%
Gemma 3 12B	16	2	0	0	0	3.7%
WizardLM 2 8x22b	8	7	0	0	0	3.1%
Z.AI GLM 4.7 Flash	16	0	0	0	0	3.1%
Claude 3.5 Sonnet	16	0	0	0	0	3.1%
GPT-4.1 Mini	16	0	0	0	0	3.1%
GPT-5.1	11	3	1	0	0	3.1%
DeepSeek V3.2	6	5	3	0	0	3.0%
Gemini 3 Flash (Preview)	14	0	0	0	0	2.9%
Nemotron 3 Super	14	0	0	0	0	2.7%
Gemini 2.5 Flash Lite (Reasoning)	9	2	0	0	0	2.1%
GPT-4.1 Nano	5	4	0	0	0	1.9%
ByteDance Seed 1.6	9	0	0	0	0	1.8%
GPT-5.4 Nano (Reasoning, Low)	9	0	0	0	0	1.8%
Ministral 3 3B	8	0	0	0	0	1.6%
Qwen 3.5 Plus (2026-02-15)	8	0	0	0	0	1.5%
Qwen 3.5 Flash	6	0	0	0	0	1.2%
Gemini 2.5 Flash Lite	3	1	0	0	0	0.8%
Gemini 3 Flash (Preview, Reasoning)	3	0	0	0	0	0.6%
Qwen 3.5 122B	2	1	0	0	0	0.5%
GPT-5 Mini	3	0	0	0	0	0.5%
Nemotron 3 Nano	2	0	0	0	0	0.3%
Inception Mercury	0	0	0	0	0	0.1%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
GPT-5.2	0	0	0	0	0	0.0%
ByteDance Seed 2.0 Mini	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-5.4 Nano (Reasoning)	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
GPT-5.4 Nano	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	93	92	79	76	88.0%
Writer: Palmyra X5	100	100	100	100	39	87.8%
Rocinante 12B	100	100	100	95	22	83.2%
GPT-5.4 (Reasoning)	100	100	88	46	45	76.0%
Cohere Command R+ (Aug. 2024)	100	100	100	64	10	74.9%
GPT-5.4	88	77	74	72	59	73.8%
Claude Opus 4.6 (Reasoning)	83	83	83	51	48	69.4%
Z.AI GLM 5	100	84	78	46	38	69.2%
Claude Sonnet 4.6	100	90	70	36	36	66.5%
GPT-5.1	87	81	67	49	43	65.3%
Grok 4	100	100	78	37	10	65.0%
Mistral Small 4 (Reasoning)	99	73	64	47	34	63.4%
Claude Opus 4.6	94	81	51	51	34	62.3%
GPT-5.4 (Reasoning, Low)	69	64	62	62	48	61.1%
Claude Opus 4	82	79	56	54	11	56.4%
Arcee AI: Trinity Large (Preview)	85	65	60	56	15	56.3%
Mistral Small Creative	75	63	57	46	38	55.9%
Claude Haiku 4.5	92	71	68	25	5	52.1%
Stealth: Healer Alpha	81	65	52	40	17	50.9%
Stealth: Hunter Alpha	89	53	48	34	27	50.2%
ByteDance Seed 2.0 Lite	80	76	52	36	2	49.2%
MiniMax M2.7	93	64	41	23	15	47.1%
MoonshotAI: Kimi K2.5	75	74	55	22	10	47.0%
Claude Opus 4.5	76	66	36	34	20	46.5%
GPT-5.4 Mini (Reasoning)	70	57	44	33	25	46.0%
Claude Sonnet 4.6 (Reasoning)	88	50	39	36	16	45.7%
GPT-5	94	67	26	22	19	45.6%
Claude 3.5 Sonnet	100	55	48	23	0	45.0%
Z.AI GLM 5 Turbo	81	67	34	32	0	42.8%
Claude Sonnet 4	83	56	40	35	0	42.8%
Grok 4.1 Fast	71	63	54	25	0	42.5%
Hermes 3 405B	90	71	42	8	0	42.3%
GPT-4.1	62	51	43	31	22	42.1%
Grok 4.20 (Beta, Reasoning)	68	60	45	28	0	40.3%
o4 Mini	100	46	34	17	2	39.9%
WizardLM 2 8x22b	65	56	31	24	23	39.8%
Mistral Large 2	72	48	34	34	8	39.4%
ByteDance Seed 2.0 Mini	98	80	18	0	0	39.2%
Grok 4 Fast	80	41	37	33	0	38.3%
DeepSeek V3.1	72	51	46	16	0	37.1%
Ministral 3 14B	54	51	34	23	23	36.8%
Mistral Medium 3.1	64	56	41	24	0	36.8%
Gemini 2.5 Flash (Reasoning)	78	49	37	20	0	36.7%
o4 Mini High	57	51	47	23	5	36.7%
MiniMax M2.5	51	45	39	26	21	36.3%
ByteDance Seed 1.6 Flash	100	45	21	15	0	36.0%
GPT-4o, May 13th (temp=0)	54	54	43	27	0	35.7%
Grok 4.20 (Beta)	52	45	35	33	15	35.6%
GPT-5.4 Mini (Reasoning, Low)	62	40	39	33	0	35.1%
DeepSeek V3.2	62	46	27	23	11	33.9%
Z.AI GLM 4.6	47	42	40	26	9	33.0%
Claude Sonnet 4.5	51	50	49	5	2	31.5%
Mistral Small 4	77	42	21	9	7	31.3%
Hermes 3 70B	74	43	36	3	0	31.2%
Gemini 2.5 Pro	35	34	31	29	24	30.6%
Gemini 2.5 Flash Lite	61	42	33	14	0	30.2%
DeepSeek V3 (2025-03-24)	42	39	31	27	11	30.0%
GPT-5.4 Mini	43	40	34	28	3	29.7%
Llama 3.1 70B	100	41	2	1	0	28.7%
Qwen 3.5 Flash	47	36	29	17	10	28.0%
Ministral 3B	65	48	27	0	0	28.0%
Claude 3.7 Sonnet	43	40	29	21	1	26.9%
Qwen 3 32B	58	33	31	5	4	26.5%
Claude 3.5 Haiku	75	34	23	0	0	26.4%
Qwen 3.5 Plus (2026-02-15)	53	27	27	23	0	26.0%
GPT-4o, Aug. 6th (temp=1)	47	42	40	1	0	25.9%
Ministral 3 8B	65	29	18	17	0	25.8%
Mistral Large	82	39	8	0	0	25.7%
Ministral 8B	60	50	17	0	0	25.4%
ByteDance Seed 1.6	53	40	17	13	0	24.6%
Gemma 3 27B	55	34	20	5	5	24.0%
Mistral Small 3.2 24B	71	31	5	0	0	21.6%
Gemini 2.5 Flash	43	24	23	19	0	21.5%
Qwen 3.5 397B A17B	53	26	25	0	0	20.8%
Gemini 3.1 Flash Lite (Preview)	55	18	15	12	0	20.1%
Aion 2.0	45	27	25	0	0	19.5%
Qwen 3.5 122B	38	29	23	6	0	19.3%
LFM2 24B	36	33	27	0	0	19.2%
Claude 3 Haiku	33	31	28	0	0	18.4%
Gemini 2.5 Flash Lite (Reasoning)	29	27	24	9	0	18.0%
Gemini 3 Pro (Preview)	42	31	16	0	0	17.7%
Gemma 3 4B	53	19	8	7	0	17.5%
Nemotron 3 Super	41	27	9	9	0	17.2%
Llama 3.1 Nemotron 70B	28	27	26	5	0	17.1%
GPT-5.4 Nano	28	25	18	9	0	16.0%
DeepSeek V3 (2024-12-26)	55	23	0	0	0	15.5%
Gemini 3 Flash (Preview)	32	23	11	11	0	15.4%
Z.AI GLM 4.7 Flash	55	13	3	2	0	14.4%
Gemma 3 12B	32	22	16	0	0	13.9%
GPT-4o Mini (temp=1)	28	26	7	5	3	13.9%
GPT-4o, Aug. 6th (temp=0)	58	7	3	0	0	13.7%
GPT-4o, May 13th (temp=1)	25	17	16	7	2	13.1%
Llama 3.1 8B	28	21	10	4	0	12.7%
GPT-5.2	23	15	12	9	0	11.9%
GPT-4.1 Nano	23	22	9	1	0	10.8%
GPT-5.4 Nano (Reasoning, Low)	16	16	16	2	0	9.8%
Inception Mercury	46	0	0	0	0	9.3%
Mistral NeMO	28	10	4	3	0	8.9%
Gemini 3 Flash (Preview, Reasoning)	18	13	7	3	2	8.9%
Mistral Large 3	28	14	2	0	0	8.8%
GPT-5.4 Nano (Reasoning)	17	12	12	2	0	8.7%
Z.AI GLM 4.7	17	11	7	0	0	6.9%
GPT-4o Mini (temp=0)	34	0	0	0	0	6.7%
DeepSeek-V2 Chat	25	8	0	0	0	6.6%
Qwen 3.5 35B	32	0	0	0	0	6.3%
Gemini 3.1 Pro (Preview)	13	9	8	0	0	6.1%
GPT-5 Mini	12	5	1	0	0	3.6%
Arcee AI: Trinity Mini	17	0	0	0	0	3.4%
GPT-4.1 Mini	14	2	0	0	0	3.1%
Ministral 3 3B	13	0	0	0	0	2.6%
GPT-5 Nano	9	2	0	0	0	2.2%
Z.AI GLM 4.5	3	0	0	0	0	0.6%
Qwen 2.5 72B	1	0	0	0	0	0.2%
Qwen 3.5 27B	0	0	0	0	0	0.0%
Qwen 3.5 9B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Rocinante 12B	100	98	96	69	0	72.6%
Mistral Small Creative	100	72	65	62	31	66.2%
Mistral Small 4 (Reasoning)	100	72	51	35	25	56.6%
Claude Opus 4	97	58	46	40	36	55.4%
Claude Opus 4.6 (Reasoning)	85	67	49	48	20	53.8%
Ministral 8B	83	47	47	42	39	51.8%
Ministral 3 14B	75	63	59	46	7	50.2%
Hermes 3 405B	100	56	33	28	25	48.4%
Qwen 3 32B	77	68	44	37	11	47.3%
Z.AI GLM 5	96	54	41	25	17	46.8%
Claude Opus 4.6	71	45	42	38	33	45.8%
Arcee AI: Trinity Large (Preview)	78	68	57	10	7	44.0%
Qwen3 235B A22B Instruct 2507	70	52	49	30	19	43.9%
Claude 3.5 Haiku	100	60	51	2	0	42.5%
Writer: Palmyra X5	82	59	40	15	12	41.4%
MiniMax M2.5	55	47	39	36	25	40.5%
MiniMax M2.7	71	48	37	27	10	38.6%
Mistral Small 4	100	71	16	5	0	38.6%
GPT-5.4 Mini	46	38	38	34	32	37.7%
GPT-5 Nano	100	83	5	0	0	37.5%
Claude Opus 4.5	58	49	31	24	23	36.9%
Llama 3.1 70B	56	45	38	34	8	36.1%
Llama 3.1 Nemotron 70B	66	45	41	17	8	35.6%
ByteDance Seed 1.6 Flash	59	41	36	28	14	35.4%
GPT-5.4 (Reasoning, Low)	41	41	39	27	26	34.7%
Ministral 3 8B	76	38	31	19	0	32.9%
Llama 3.1 8B	69	69	12	5	0	31.2%
Z.AI GLM 5 Turbo	90	42	13	10	0	31.0%
Claude 3 Haiku	45	45	35	23	5	30.7%
o4 Mini High	50	43	32	19	0	28.9%
GPT-5.4 (Reasoning)	40	31	31	25	13	28.0%
Mistral Large 3	77	44	17	0	0	27.8%
GPT-5.4 Mini (Reasoning)	51	47	21	20	0	27.6%
Hermes 3 70B	71	31	21	12	0	27.2%
Grok 4 Fast	66	31	24	14	0	27.2%
Grok 4.20 (Beta, Reasoning)	59	42	26	0	0	25.5%
DeepSeek V3 (2024-12-26)	63	29	27	6	0	25.1%
ByteDance Seed 2.0 Lite	45	36	22	18	0	24.2%
Z.AI GLM 4.6	64	24	19	11	0	23.6%
Grok 4.1 Fast	35	35	32	8	8	23.5%
MoonshotAI: Kimi K2.5	49	39	16	8	0	22.3%
GPT-5.4	39	36	28	6	0	21.7%
Claude Sonnet 4.6 (Reasoning)	49	23	18	12	0	20.5%
GPT-4.1 Nano	33	23	17	16	12	20.2%
Gemini 3.1 Flash Lite (Preview)	39	25	21	17	0	20.1%
GPT-4o, Aug. 6th (temp=1)	37	30	26	7	0	20.1%
o4 Mini	43	28	27	0	0	19.8%
Claude Sonnet 4.5	56	28	14	0	0	19.7%
Gemma 3 27B	31	24	22	20	2	19.5%
WizardLM 2 8x22b	79	11	6	0	0	19.3%
Claude Haiku 4.5	50	23	15	6	0	18.8%
GPT-4o Mini (temp=1)	46	17	16	10	2	18.3%
DeepSeek V3 (2025-03-24)	35	25	20	10	0	18.0%
GPT-5	45	43	2	0	0	18.0%
DeepSeek V3.1	66	17	5	0	0	17.6%
Qwen 3.5 397B A17B	26	23	21	17	0	17.4%
DeepSeek V3.2	30	30	24	0	0	16.7%
GPT-4o, May 13th (temp=1)	27	22	17	17	0	16.5%
GPT-5.1	42	14	9	8	7	16.0%
Grok 4.20 (Beta)	34	27	16	1	0	15.6%
Mistral Small 3.2 24B	50	18	9	0	0	15.5%
Gemma 3 12B	32	23	16	5	0	15.4%
Stealth: Healer Alpha	48	14	11	0	0	14.6%
Claude Sonnet 4	28	27	11	3	2	14.2%
Mistral Large	29	25	16	0	0	13.9%
Mistral Medium 3.1	26	22	21	0	0	13.7%
Cohere Command R+ (Aug. 2024)	34	19	16	0	0	13.6%
ByteDance Seed 2.0 Mini	55	5	4	0	0	12.8%
Gemini 2.5 Flash Lite	31	18	11	3	0	12.7%
Gemini 2.5 Pro	39	18	6	0	0	12.6%
Grok 4	21	14	13	7	5	12.0%
GPT-5.4 Mini (Reasoning, Low)	34	11	8	6	0	11.7%
Gemini 2.5 Flash Lite (Reasoning)	17	16	14	9	0	11.4%
GPT-4.1 Mini	30	26	1	0	0	11.3%
Mistral Large 2	57	0	0	0	0	11.3%
Stealth: Hunter Alpha	26	14	12	3	0	11.1%
GPT-4o, Aug. 6th (temp=0)	39	10	5	0	0	10.9%
Qwen 3.5 35B	31	23	0	0	0	10.8%
GPT-4.1	22	16	14	2	0	10.7%
Mistral NeMO	42	11	1	0	0	10.7%
Gemini 3 Pro (Preview)	41	10	0	0	0	10.1%
Qwen 3.5 122B	26	18	1	0	0	9.0%
Qwen 3.5 Flash	38	5	2	0	0	9.0%
Claude Sonnet 4.6	17	15	13	0	0	8.9%
Claude 3.5 Sonnet	16	13	11	0	0	7.9%
Z.AI GLM 4.5	26	11	0	0	0	7.3%
Aion 2.0	16	15	3	2	0	7.2%
Gemini 2.5 Flash	34	0	0	0	0	6.7%
Claude 3.7 Sonnet	16	14	0	0	0	5.9%
Z.AI GLM 4.7	29	0	0	0	0	5.7%
Gemma 3 4B	16	9	0	0	0	4.9%
GPT-4o, May 13th (temp=0)	10	7	0	0	0	3.5%
ByteDance Seed 1.6	13	3	0	0	0	3.1%
DeepSeek-V2 Chat	15	0	0	0	0	2.9%
Gemini 2.5 Flash (Reasoning)	13	0	0	0	0	2.6%
GPT-4o Mini (temp=0)	13	0	0	0	0	2.5%
Ministral 3 3B	12	0	0	0	0	2.3%
GPT-5.2	11	0	0	0	0	2.2%
LFM2 24B	8	1	0	0	0	1.9%
Arcee AI: Trinity Mini	9	0	0	0	0	1.8%
Qwen 3.5 9B	6	0	0	0	0	1.2%
Nemotron 3 Super	3	2	0	0	0	1.0%
Qwen 2.5 72B	4	0	0	0	0	0.8%
Ministral 3B	2	0	0	0	0	0.5%
GPT-5.4 Nano	1	0	0	0	0	0.3%
Z.AI GLM 4.7 Flash	1	0	0	0	0	0.2%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
GPT-5 Mini	0	0	0	0	0	0.0%
Qwen 3.5 27B	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview, Reasoning)	0	0	0	0	0	0.0%
Qwen 3.5 Plus (2026-02-15)	0	0	0	0	0	0.0%
Gemini 3 Flash (Preview)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-5.4 Nano (Reasoning)	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%
GPT-5.4 Nano (Reasoning, Low)	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	96	89	62	89.4%
Writer: Palmyra X5	100	100	100	81	61	88.3%
Z.AI GLM 5	100	99	96	90	54	87.8%
Hermes 3 70B	100	100	93	72	71	87.3%
Rocinante 12B	100	100	71	58	55	76.7%
Claude Haiku 4.5	100	100	75	60	25	71.9%
Llama 3.1 8B	100	100	65	62	31	71.8%
Grok 4.1 Fast	100	83	78	57	40	71.6%
Claude Sonnet 4.5	89	75	75	57	49	69.1%
Arcee AI: Trinity Large (Preview)	87	87	59	53	53	67.7%
GPT-5.4	86	76	64	54	51	66.0%
GPT-4.1 Nano	91	73	68	46	45	64.4%
Claude Opus 4.6	100	79	73	39	29	64.0%
Claude Opus 4.5	88	73	72	44	41	63.7%
MiniMax M2.5	73	69	58	54	48	60.6%
Mistral Small 4 (Reasoning)	100	84	82	27	9	60.5%
GPT-5.4 (Reasoning, Low)	79	68	64	54	34	59.6%
Claude Opus 4.6 (Reasoning)	79	67	50	50	44	58.2%
Mistral Small 4	97	84	64	31	14	57.8%
Claude Sonnet 4.6	85	79	55	47	19	57.1%
MoonshotAI: Kimi K2.5	94	89	55	28	16	56.4%
GPT-5.4 (Reasoning)	77	74	48	44	39	56.4%
Claude 3 Haiku	83	82	67	19	10	52.3%
Hermes 3 405B	70	66	49	47	23	51.0%
Ministral 3 14B	87	68	47	31	20	50.4%
Gemini 2.5 Flash Lite	78	61	42	38	23	48.2%
GPT-4o, Aug. 6th (temp=1)	69	56	40	38	36	47.9%
Mistral Small Creative	82	56	39	39	20	47.3%
Claude Opus 4	63	54	41	39	36	46.7%
Gemini 2.5 Flash Lite (Reasoning)	77	41	39	35	34	45.3%
Cohere Command R+ (Aug. 2024)	71	65	42	28	15	44.4%
GPT-4.1	56	45	44	37	36	43.7%
Claude 3.7 Sonnet	59	49	47	39	21	42.9%
GPT-5.4 Mini	71	46	41	36	20	42.8%
Llama 3.1 70B	90	48	35	23	13	41.8%
Mistral Small 3.2 24B	85	65	30	29	0	41.7%
Llama 3.1 Nemotron 70B	65	50	37	30	28	41.7%
GPT-4o, May 13th (temp=1)	67	63	41	35	0	41.2%
GPT-4.1 Mini	74	57	48	21	2	40.5%
Z.AI GLM 4.6	50	42	40	37	32	40.4%
Mistral Medium 3.1	66	51	31	28	23	39.9%
Grok 4.20 (Beta)	61	45	43	27	24	39.8%
Z.AI GLM 5 Turbo	57	55	42	35	8	39.5%
Gemma 3 27B	62	56	38	24	15	39.0%
Gemma 3 12B	61	49	44	21	17	38.6%
Grok 4 Fast	68	42	38	22	22	38.5%
Stealth: Hunter Alpha	65	65	35	26	0	38.1%
Claude 3.5 Haiku	73	51	47	10	8	37.7%
Grok 4	63	46	42	20	13	36.9%
GPT-5.4 Mini (Reasoning, Low)	48	47	36	31	18	36.1%
Claude Sonnet 4	100	55	14	10	0	35.8%
DeepSeek V3.1	57	49	44	15	7	34.6%
ByteDance Seed 1.6 Flash	60	52	34	18	8	34.4%
Gemini 3.1 Pro (Preview)	74	47	32	13	5	34.3%
Claude Sonnet 4.6 (Reasoning)	49	43	31	26	21	34.2%
GPT-5.1	48	39	38	26	18	33.7%
Aion 2.0	40	40	30	30	18	31.4%
o4 Mini High	50	38	26	21	14	29.7%
Ministral 3 8B	96	26	14	12	0	29.6%
Grok 4.20 (Beta, Reasoning)	42	37	28	21	17	29.0%
Qwen 3.5 Plus (2026-02-15)	59	43	23	19	0	28.8%
DeepSeek-V2 Chat	43	40	34	16	3	27.2%
Stealth: Healer Alpha	63	42	15	11	3	26.8%
Z.AI GLM 4.7 Flash	43	30	27	19	13	26.5%
WizardLM 2 8x22b	46	31	25	24	6	26.5%
Gemini 2.5 Pro	51	29	22	14	12	25.6%
Claude 3.5 Sonnet	43	42	36	7	0	25.5%
GPT-4o, Aug. 6th (temp=0)	39	37	22	20	8	25.1%
Ministral 3 3B	43	35	24	12	8	24.6%
Gemini 3 Pro (Preview)	46	34	30	9	2	24.3%
DeepSeek V3.2	41	26	23	19	12	24.2%
MiniMax M2.7	47	31	17	13	12	24.0%
Z.AI GLM 4.5	42	27	25	16	11	24.0%
o4 Mini	51	33	16	15	5	23.8%
GPT-5.4 Mini (Reasoning)	41	30	29	14	5	23.8%
GPT-5 Nano	100	14	0	0	0	22.9%
Mistral Large	70	17	16	8	1	22.5%
GPT-4o Mini (temp=1)	40	22	19	19	11	22.3%
Gemini 2.5 Flash	35	28	27	19	1	21.9%
Mistral Large 3	36	29	25	11	8	21.8%
Gemini 2.5 Flash (Reasoning)	38	34	21	15	0	21.5%
Ministral 8B	58	31	10	1	0	19.9%
Gemma 3 4B	26	25	14	14	14	18.7%
Mistral NeMO	33	28	17	15	0	18.6%
DeepSeek V3 (2025-03-24)	34	31	17	2	0	16.9%
ByteDance Seed 2.0 Mini	30	30	24	0	0	16.8%
Arcee AI: Trinity Mini	28	23	16	14	0	16.2%
Gemini 3.1 Flash Lite (Preview)	40	26	7	2	0	15.2%
LFM2 24B	36	20	11	8	0	15.1%
ByteDance Seed 2.0 Lite	39	14	11	9	2	15.0%
Qwen 3.5 397B A17B	35	35	5	0	0	14.9%
Nemotron 3 Nano	49	18	7	0	0	14.9%
Z.AI GLM 4.7	28	15	11	8	5	13.4%
Qwen 3.5 35B	30	25	9	0	0	12.9%
GPT-5.4 Nano	33	24	4	1	1	12.6%
Mistral Large 2	31	16	9	8	0	12.6%
GPT-5 Mini	37	16	5	4	0	12.5%
GPT-4o, May 13th (temp=0)	28	20	12	0	0	12.0%
Ministral 3B	27	21	11	2	0	12.0%
Qwen 3.5 Flash	38	15	6	0	0	11.7%
Qwen 3 32B	21	19	10	0	0	9.9%
ByteDance Seed 1.6	41	8	0	0	0	9.9%
GPT-5.4 Nano (Reasoning, Low)	19	15	7	4	3	9.4%
GPT-5.4 Nano (Reasoning)	24	9	0	0	0	6.4%
Gemini 3 Flash (Preview)	12	10	4	3	0	5.9%
Qwen 2.5 72B	15	11	1	0	0	5.1%
Qwen 3.5 9B	19	0	0	0	0	3.8%
DeepSeek V3 (2024-12-26)	18	0	0	0	0	3.6%
Gemini 3 Flash (Preview, Reasoning)	13	1	0	0	0	2.8%
Nemotron 3 Super	12	0	0	0	0	2.3%
GPT-4o Mini (temp=0)	11	0	0	0	0	2.2%
GPT-5	11	0	0	0	0	2.2%
GPT-5.2	9	0	0	0	0	1.8%
Qwen 3.5 27B	5	0	0	0	0	1.1%
Qwen 3.5 122B	3	1	0	0	0	0.8%
Inception Mercury 2	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Inception Mercury	0	0	0	0	0	0.0%

Subject-first sentence starts

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets