Adverb-first sentence starts

Test: Bad Writing Habits

Avg. Score

54.7%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Writer: Palmyra X5	94.5%	$0.011	22.0s	64%
2	Qwen3 235B A22B Instruct 2507	90.8%	$0.0011	59.2s	56%
3	Mistral Small 4 (Reasoning)	86.4%	$0.0022	30.2s	54%
4	Mistral Medium 3.1	83.9%	$0.0048	36.5s	51%
5	GPT-5.4 Mini	85.2%	$0.015	16.8s	49%
6	Ministral 3 14B	81.0%	$0.0007	11.7s	43%
7	GPT-5.4 Mini (Reasoning, Low)	79.5%	$0.015	16.8s	45%
8	GPT-5.4 Nano	76.8%	$0.0057	26.3s	44%
9	GPT-5.4 Nano (Reasoning, Low)	76.7%	$0.0055	20.6s	42%
10	GPT-5.4	87.5%	$0.049	1.4m	58%
11	GPT-5.4 Mini (Reasoning)	79.2%	$0.022	28.1s	42%
12	Mistral Small 4	76.4%	$0.0014	18.2s	34%
13	GPT-5.4 (Reasoning, Low)	86.9%	$0.055	1.4m	54%
14	Mistral Large 3	77.9%	$0.0033	30.3s	33%
15	Mistral Large 2	78.0%	$0.013	29.4s	33%
16	GPT-5.4 Nano (Reasoning)	70.5%	$0.0061	24.5s	32%
17	Gemma 3 4B	71.3%	$0.0002	20.0s	28%
18	GPT-4.1 Nano	71.8%	$0.0007	13.3s	26%
19	Grok 4.20	72.5%	$0.0093	45.7s	31%
20	Ministral 8B	70.1%	$0.0004	10.4s	22%
21	Ministral 3 8B	69.8%	$0.0008	19.6s	22%
22	Gemma 3 27B	67.9%	$0.0006	52.6s	25%
23	MiniMax M2.5	69.8%	$0.0034	1.3m	25%
24	Claude Haiku 4.5	64.8%	$0.011	21.6s	20%
25	Z.AI GLM 5	69.5%	$0.0084	1.2m	24%
26	Claude Sonnet 4.5	71.0%	$0.035	38.1s	26%
27	Z.AI GLM 5 Turbo	65.3%	$0.0081	33.2s	21%
28	Claude Sonnet 5 (Reasoning, Low)	73.4%	$0.031	38.4s	23%
29	Gemini 2.5 Flash Lite (Reasoning)	58.0%	$0.0028	30.8s	23%
30	Gemma 3 12B	65.3%	$0.0004	41.3s	18%
31	Z.AI GLM 5.2 (Reasoning, High)	67.3%	$0.011	1.0m	22%
32	Grok 4.20 (Reasoning)	67.9%	$0.018	1.5m	28%
33	Ministral 3 3B	60.5%	$0.0005	11.1s	15%
34	Ministral 3B	61.2%	$0.0001	8.1s	13%
35	MiniMax M2.7	65.6%	$0.0040	1.1m	20%
36	DeepSeek V4 Flash (Reasoning)	59.4%	$0.0007	31.1s	16%
37	DeepSeek V3 (2025-03-24)	63.3%	$0.0014	39.4s	14%
38	Claude Opus 4.7	72.6%	$0.069	30.4s	26%
39	DeepSeek V4 Flash	55.0%	$0.0006	31.6s	16%
40	Xiaomi MIMO v2.5	53.1%	$0.0054	31.8s	19%
41	Grok 4.5 (Reasoning, Low)	64.9%	$0.018	1.1m	19%
42	Gemini 2.5 Flash Lite	54.3%	$0.0009	9.5s	12%
43	GPT-5.4 (Reasoning)	79.8%	$0.089	2.6m	44%
44	GPT-4.1	59.6%	$0.018	44.7s	18%
45	Z.AI GLM 5.1	63.6%	$0.014	1.5m	22%
46	ByteDance Seed 1.6 Flash	53.3%	$0.0013	27.3s	14%
47	Claude Sonnet 5	66.5%	$0.027	33.5s	14%
48	Grok 4.5 (Reasoning, High)	69.5%	$0.030	1.6m	23%
49	GPT-5.5 (Reasoning, Low)	82.2%	$0.139	1.8m	49%
50	o4 Mini	53.5%	$0.015	25.7s	17%
51	Xiaomi MIMO v2.5 Pro	59.2%	$0.0085	53.5s	16%
52	GPT-5.1	69.8%	$0.054	1.8m	31%
53	Qwen 3 32B	57.3%	$0.0015	54.6s	15%
54	o4 Mini High	57.3%	$0.025	47.2s	20%
55	Gemini 2.5 Flash	52.2%	$0.0052	10.6s	10%
56	Aion 3.0 Mini	58.2%	$0.0053	1.2m	15%
57	GPT-5 Nano	55.1%	$0.0042	1.4m	18%
58	Aion 3.0	58.8%	$0.024	1.0m	18%
59	Mistral NeMO	50.3%	$0.0005	10.1s	8%
60	Claude Sonnet 4.6 (Reasoning)	67.0%	$0.060	1.2m	24%
61	Cydonia 24B V4.1	54.6%	$0.0014	44.8s	11%
62	Claude Opus 4.7 (Reasoning)	68.0%	$0.076	32.0s	22%
63	DeepSeek-V2 Chat	56.8%	$0.0021	53.3s	10%
64	Qwen 3.6 Flash	50.8%	$0.010	41.4s	15%
65	Claude Sonnet 5 (Reasoning)	63.0%	$0.030	38.9s	12%
66	GPT-4.1 Mini	49.8%	$0.0027	19.0s	10%
67	Gemini 2.5 Flash (Reasoning)	51.4%	$0.011	21.5s	11%
68	MiniMax M3	68.2%	$0.0060	3.1m	25%
69	GPT-5.5	78.7%	$0.139	1.7m	44%
70	Claude Opus 4.8 (Reasoning, Low)	67.9%	$0.071	41.9s	21%
71	Z.AI GLM 4.7 Flash	50.4%	$0.0017	1.2m	17%
72	GPT-4o, Aug. 6th (temp=1)	54.9%	$0.018	24.4s	10%
73	DeepSeek V4 Pro	54.1%	$0.0048	1.3m	14%
74	GPT-4o Mini (temp=1)	50.4%	$0.0012	34.8s	9%
75	GPT-5.5 (Reasoning)	78.2%	$0.142	1.8m	44%
76	Gemini 3.5 Flash (Reasoning, Minimal)	50.3%	$0.018	12.0s	10%
77	Qwen 3.5 Plus (2026-02-15)	45.0%	$0.0060	31.5s	12%
78	Claude Sonnet 4	54.7%	$0.032	43.7s	15%
79	Gemini 3 Flash (Preview)	41.4%	$0.0078	19.6s	12%
80	Gemini 3.1 Flash Lite (Preview)	36.7%	$0.0030	8.4s	12%
81	Claude Opus 4.8 (Reasoning)	63.5%	$0.071	41.7s	19%
82	Z.AI GLM 4.6	48.7%	$0.0065	51.5s	11%
83	DeepSeek V3 (2024-12-26)	50.8%	$0.0021	54.6s	8%
84	Gemma 4 26B	43.2%	$0.0009	55.1s	13%
85	Claude Opus 4.5	63.6%	$0.070	53.4s	19%
86	Claude Opus 4.6	65.2%	$0.078	1.2m	24%
87	Hermes 3 70B	53.2%	$0.0010	1.2m	9%
88	Gemini 3.1 Flash Lite	39.5%	$0.0030	12.1s	9%
89	Claude Sonnet 4.6	56.6%	$0.031	39.3s	9%
90	Gemini 3.1 Flash Lite (Reasoning)	36.6%	$0.0030	11.9s	10%
91	Qwen 3.6 35B	47.0%	$0.0083	1.0m	12%
92	Qwen 3.5 Flash	36.6%	$0.0025	47.5s	14%
93	GPT-5.2	59.9%	$0.056	1.5m	20%
94	Grok 4.3	39.2%	$0.0069	30.5s	9%
95	Aion 2.0	46.1%	$0.0064	1.3m	11%
96	Gemini 3 Flash (Preview, Reasoning)	39.6%	$0.012	30.1s	10%
97	Hermes 3 405B	43.6%	$0.0032	53.2s	8%
98	DeepSeek V3.2	50.1%	$0.0014	1.9m	12%
99	DeepSeek V4 Pro (Reasoning)	62.0%	$0.015	3.1m	18%
100	Z.AI GLM 4.5	38.7%	$0.0051	42.1s	8%
101	Z.AI GLM 4.7	44.2%	$0.010	1.4m	12%
102	Z.AI GLM 4.5 Air	40.7%	$0.0029	58.2s	7%
103	Gemini 2.5 Pro	48.0%	$0.036	36.2s	9%
104	Claude Opus 4.6 (Reasoning)	62.8%	$0.088	1.4m	21%
105	GPT-5 Mini	39.9%	$0.0100	57.4s	10%
106	Qwen 3.5 397B A17B	54.6%	$0.014	3.0m	20%
107	Gemma 4 26B (Reasoning)	47.1%	$0.0013	2.0m	11%
108	Cohere Command R+ (Aug. 2024)	42.4%	$0.020	52.5s	8%
109	Qwen 3.5 35B	36.5%	$0.018	1.0m	13%
110	Arcee AI: Trinity Mini	33.6%	$0.0003	9.2s	0%
111	Nemotron 3 Super	36.7%	$0.0000	1.4m	10%
112	DeepSeek V3.1	43.2%	$0.0020	1.8m	10%
113	WizardLM 2 8x22b	42.8%	$0.0026	1.8m	9%
114	Qwen 3.5 122B	33.5%	$0.025	1.1m	13%
115	Gemma 4 31B	36.3%	$0.0010	1.6m	8%
116	Qwen 3.5 Plus (2026-04-20)	41.7%	$0.017	1.8m	10%
117	Gemini 3.5 Flash (Reasoning)	44.2%	$0.071	37.6s	10%
118	Llama 3.1 70B	26.1%	$0.0015	29.4s	0%
119	GPT-4o Mini (temp=0)	25.8%	$0.0012	34.8s	0%
120	Qwen 2.5 72B	25.0%	$0.0010	36.7s	0%
121	Gemma 4 31B (Reasoning)	30.0%	$0.0014	2.2m	10%
122	MoonshotAI: Kimi K2.5	47.1%	$0.019	3.2m	10%
123	GPT-4o, Aug. 6th (temp=0)	23.1%	$0.023	22.7s	0%
124	Qwen 3.5 27B	24.4%	$0.020	1.6m	10%
125	Qwen 3.6 27B	39.0%	$0.025	2.3m	8%
126	Qwen 3.5 9B	23.3%	$0.0011	1.4m	2%
127	Inception Mercury 2	5.0%	$0.0032	7.0s	0%
128	Nemotron 3 Nano	15.3%	$0.0010	1.1m	0%
129	GPT-5	40.7%	$0.065	2.8m	17%
130	Qwen3.6 Max Preview	46.5%	$0.050	3.5m	12%
131	ByteDance Seed 2.0 Lite	24.7%	$0.012	2.2m	0%
132	Claude Opus 4	65.4%	$0.209	1.4m	23%
133	GPT-OSS 120B	10.6%	$0.0015	1.8m	0%
134	Grok 4.3 (Reasoning)	25.5%	$0.021	2.3m	0%
135	ByteDance Seed 2.0 Mini	39.5%	$0.0045	4.9m	9%
136	Qwen3.7 Max	32.1%	$0.068	2.3m	7%
137	ByteDance Seed 1.6	20.3%	$0.013	2.5m	0%
138	Gemini 3.1 Pro (Preview)	28.9%	$0.107	1.8m	7%
139	MoonshotAI: Kimi K2.6	56.4%	$0.058	6.5m	17%
140	Mistral Small 3.2 24B	19.1%	$0.0069	5.7m	0%
54.72%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	93	98.5%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	100	83	96.7%
Claude Sonnet 4.6	100	100	98	98	82	95.7%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	76	95.2%
GPT-5.4	100	100	100	100	69	93.8%
GPT-5.4 (Reasoning)	100	100	100	90	78	93.7%
Qwen3 235B A22B Instruct 2507	100	100	100	100	50	90.0%
GPT-5.4 Mini (Reasoning)	100	100	100	78	70	89.6%
Cydonia 24B V4.1	100	100	100	100	46	89.1%
GPT-5.4 (Reasoning, Low)	100	100	100	100	40	88.0%
Gemma 3 4B	100	100	100	88	46	86.6%
Z.AI GLM 5	100	100	100	80	40	84.0%
GPT-5.5 (Reasoning, Low)	100	96	87	70	59	82.5%
Claude Opus 4.6	100	100	73	72	67	82.3%
GPT-5.4 Nano	100	100	94	72	44	82.2%
DeepSeek V4 Flash (Reasoning)	100	91	88	81	49	82.0%
GPT-5.4 Nano (Reasoning)	100	100	100	56	54	82.0%
GPT-5.5	100	97	92	67	53	81.6%
Grok 4.5 (Reasoning, High)	100	100	100	100	0	80.0%
DeepSeek-V2 Chat	100	100	100	100	0	80.0%
Claude Opus 4.7	100	100	100	52	47	79.8%
Claude Opus 4	100	100	71	64	62	79.3%
WizardLM 2 8x22b	100	100	83	81	28	78.4%
Z.AI GLM 5.2 (Reasoning, High)	100	100	100	89	0	77.9%
GPT-4o, Aug. 6th (temp=1)	100	100	71	64	51	77.3%
Claude Sonnet 5	100	100	100	76	0	75.2%
Hermes 3 405B	100	100	100	71	0	74.2%
Gemini 2.5 Flash	100	98	94	78	0	73.9%
Claude Sonnet 5 (Reasoning)	100	100	100	65	0	73.1%
GPT-5.4 Mini	100	100	98	60	0	71.5%
MoonshotAI: Kimi K2.5	100	93	67	49	48	71.3%
GPT-5 Nano	100	97	84	42	32	70.9%
Z.AI GLM 4.7 Flash	100	100	79	40	35	70.8%
GPT-5.4 Nano (Reasoning, Low)	100	71	66	59	52	69.7%
Mistral Small 4 (Reasoning)	100	95	59	48	46	69.5%
DeepSeek V4 Pro (Reasoning)	100	100	69	40	39	69.4%
DeepSeek V3 (2024-12-26)	100	90	81	72	0	68.8%
Gemma 3 12B	100	100	100	40	0	67.9%
GPT-5.2	100	97	59	43	34	66.8%
Claude Opus 4.5	100	100	100	30	0	66.0%
MiniMax M2.7	100	85	85	56	0	65.3%
Claude Opus 4.8 (Reasoning)	95	61	57	54	54	64.2%
Grok 4.5 (Reasoning, Low)	100	100	80	40	0	64.0%
Aion 2.0	100	100	69	51	0	64.0%
GPT-5.5 (Reasoning)	79	72	58	56	51	63.6%
Gemini 3 Flash (Preview, Reasoning)	78	75	68	57	38	63.3%
Claude Sonnet 4.6 (Reasoning)	100	95	76	44	0	63.0%
Gemini 2.5 Flash Lite (Reasoning)	84	82	57	43	40	61.5%
DeepSeek V4 Flash	100	61	52	44	44	60.1%
GPT-5.1	89	72	72	50	16	59.9%
Claude Opus 4.8 (Reasoning, Low)	100	100	52	45	0	59.4%
Ministral 3B	100	100	95	0	0	59.0%
Z.AI GLM 5 Turbo	100	78	43	42	33	58.9%
Claude Opus 4.6 (Reasoning)	100	97	38	35	24	58.9%
MiniMax M2.5	100	87	55	52	0	58.7%
Gemini 2.5 Flash Lite	100	100	91	0	0	58.3%
Ministral 3 14B	100	100	87	0	0	57.3%
Ministral 8B	100	100	82	0	0	56.5%
Z.AI GLM 4.6	100	91	51	38	0	56.1%
Qwen 2.5 72B	99	88	87	0	0	54.7%
GPT-4.1 Nano	100	63	56	54	0	54.3%
MiniMax M3	100	69	61	40	0	53.9%
Mistral Large 3	91	79	53	40	0	52.7%
Hermes 3 70B	100	95	67	0	0	52.4%
Claude Sonnet 4	100	65	54	42	0	52.2%
Cohere Command R+ (Aug. 2024)	90	71	51	46	0	51.6%
Ministral 3 8B	100	100	57	0	0	51.5%
Qwen3.6 Max Preview	100	74	51	32	0	51.2%
Gemini 2.5 Flash (Reasoning)	100	82	37	34	0	50.7%
Claude Opus 4.7 (Reasoning)	100	100	52	0	0	50.4%
Aion 3.0 Mini	100	100	51	0	0	50.3%
Z.AI GLM 5.1	100	79	69	0	0	49.6%
Aion 3.0	100	100	47	0	0	49.4%
Qwen 3.5 397B A17B	100	74	55	14	0	48.8%
Gemini 2.5 Pro	94	72	42	29	0	47.6%
DeepSeek V3 (2025-03-24)	100	88	49	0	0	47.3%
Qwen 3 32B	93	49	46	45	0	46.6%
Mistral Large 2	100	100	32	0	0	46.5%
GPT-4.1 Mini	85	50	49	46	0	46.0%
Claude Haiku 4.5	100	79	51	0	0	46.0%
GPT-4.1	98	66	32	31	0	45.5%
Gemma 4 26B	80	69	39	38	0	45.3%
Mistral Small 3.2 24B	100	80	46	0	0	45.3%
MoonshotAI: Kimi K2.6	100	57	37	30	0	44.8%
Mistral NeMO	74	67	44	33	0	43.4%
Qwen 3.6 35B	100	66	51	0	0	43.4%
Z.AI GLM 4.5 Air	81	67	35	33	0	43.3%
Mistral Small 4	100	78	38	0	0	43.1%
Grok 4.3 (Reasoning)	65	60	50	35	0	41.9%
Gemma 3 27B	79	46	43	40	0	41.7%
ByteDance Seed 2.0 Mini	100	62	46	0	0	41.5%
Gemini 3 Flash (Preview)	67	61	47	29	0	40.8%
Qwen3.7 Max	74	57	39	31	0	40.4%
GPT-4o Mini (temp=1)	100	97	0	0	0	39.3%
Xiaomi MIMO v2.5	100	69	26	0	0	39.0%
DeepSeek V3.2	69	65	33	28	0	38.9%
Qwen 3.5 Plus (2026-04-20)	65	46	44	22	17	38.8%
Mistral Medium 3.1	61	39	34	32	25	38.1%
o4 Mini High	78	44	43	22	0	37.3%
Gemini 3.1 Flash Lite	100	44	37	0	0	36.4%
Grok 4.20	66	47	25	23	20	36.1%
Grok 4.3	93	43	36	0	0	34.4%
Z.AI GLM 4.7	70	41	32	28	0	34.3%
Xiaomi MIMO v2.5 Pro	100	43	27	0	0	34.0%
ByteDance Seed 2.0 Lite	100	68	0	0	0	33.6%
o4 Mini	48	45	42	31	0	33.4%
Qwen 3.6 27B	100	42	25	0	0	33.4%
Qwen 3.5 Plus (2026-02-15)	65	35	32	31	0	32.7%
ByteDance Seed 1.6 Flash	66	55	35	0	0	31.3%
GPT-5 Mini	89	23	23	20	0	31.0%
Nemotron 3 Super	46	44	40	0	0	26.1%
DeepSeek V4 Pro	85	43	0	0	0	25.6%
Qwen 3.6 Flash	48	44	34	0	0	25.1%
Grok 4.20 (Reasoning)	59	39	26	0	0	24.8%
GPT-5	57	28	21	14	0	24.0%
Qwen 3.5 Flash	45	34	19	16	0	22.7%
GPT-4o, Aug. 6th (temp=0)	63	47	0	0	0	22.0%
Gemma 4 26B (Reasoning)	52	50	0	0	0	20.4%
Gemini 3.5 Flash (Reasoning)	100	0	0	0	0	20.0%
Gemini 3.5 Flash (Reasoning, Minimal)	100	0	0	0	0	20.0%
Ministral 3 3B	100	0	0	0	0	20.0%
Qwen 3.5 122B	41	31	26	0	0	19.6%
Qwen 3.5 27B	47	25	19	0	0	18.2%
Gemini 3.1 Flash Lite (Preview)	48	43	0	0	0	18.2%
Qwen 3.5 35B	48	25	18	0	0	18.2%
ByteDance Seed 1.6	90	0	0	0	0	18.0%
GPT-4o Mini (temp=0)	43	41	0	0	0	16.8%
Llama 3.1 70B	74	0	0	0	0	14.8%
Arcee AI: Trinity Mini	72	0	0	0	0	14.5%
DeepSeek V3.1	37	33	0	0	0	14.1%
GPT-OSS 120B	47	23	0	0	0	14.0%
Gemma 4 31B (Reasoning)	56	0	0	0	0	11.3%
Gemma 4 31B	54	0	0	0	0	10.8%
Gemini 3.1 Pro (Preview)	31	22	0	0	0	10.6%
Z.AI GLM 4.5	43	0	0	0	0	8.7%
Nemotron 3 Nano	21	0	0	0	0	4.1%
Qwen 3.5 9B	17	0	0	0	0	3.4%
Gemini 3.1 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	100	100	100.0%
Grok 4.5 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 5 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 5	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Aion 3.0 Mini	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Z.AI GLM 4.5 Air	100	100	100	99	95	98.8%
GPT-4o Mini (temp=1)	100	100	100	100	91	98.3%
Gemma 3 12B	100	100	100	100	89	97.9%
GPT-5.5 (Reasoning, Low)	100	100	100	100	89	97.8%
Mistral Small 4 (Reasoning)	100	100	100	100	86	97.2%
Grok 4.20	100	100	100	100	85	96.9%
GPT-5.4 Mini	100	100	100	100	84	96.8%
Gemini 2.5 Flash	100	100	100	100	83	96.5%
GPT-4.1 Nano	100	100	100	100	81	96.3%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	99	80	95.8%
MoonshotAI: Kimi K2.6	100	100	100	95	83	95.7%
MiniMax M2.7	100	100	100	100	78	95.5%
Ministral 3 14B	100	100	100	100	77	95.3%
GPT-5.4	100	100	100	100	76	95.2%
Z.AI GLM 4.7	100	100	100	100	75	95.0%
GPT-5.4 Nano	100	100	100	100	71	94.1%
GPT-5.4 Mini (Reasoning)	100	100	100	100	70	94.0%
GPT-5.5 (Reasoning)	100	100	100	98	70	93.7%
Mistral Medium 3.1	100	100	100	100	68	93.6%
Ministral 3 3B	100	100	100	100	67	93.3%
WizardLM 2 8x22b	100	100	100	91	71	92.3%
GPT-4.1 Mini	100	100	100	80	80	92.1%
GPT-5.5	100	100	100	90	64	90.7%
Aion 2.0	100	100	100	80	69	89.8%
Claude Sonnet 4.5	100	100	100	81	67	89.7%
Z.AI GLM 4.5	100	100	100	100	48	89.5%
Grok 4.5 (Reasoning, High)	100	100	100	78	69	89.4%
Claude Sonnet 4	100	100	100	100	47	89.4%
DeepSeek V4 Flash (Reasoning)	100	100	100	74	72	89.3%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	46	89.3%
DeepSeek-V2 Chat	100	100	100	88	58	89.2%
Claude Opus 4	100	100	100	81	62	88.7%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	83	60	88.6%
GPT-5.4 Nano (Reasoning)	100	100	92	77	72	88.3%
Z.AI GLM 4.6	100	100	100	100	38	87.7%
GPT-5 Nano	100	100	100	70	68	87.5%
Claude Opus 4.7	100	100	82	78	75	87.1%
DeepSeek V4 Pro	100	100	100	100	34	86.8%
Claude Opus 4.8 (Reasoning)	100	100	100	94	38	86.4%
DeepSeek V4 Pro (Reasoning)	100	100	100	93	35	85.6%
Grok 4.20 (Reasoning)	100	100	100	78	50	85.5%
Claude Sonnet 4.6	100	100	100	78	46	84.9%
DeepSeek V3 (2024-12-26)	100	100	100	71	53	84.8%
Claude Opus 4.7 (Reasoning)	100	100	100	78	41	83.6%
GPT-5.1	100	100	90	67	60	83.6%
Gemini 2.5 Flash Lite	100	100	98	85	32	83.1%
Qwen3.6 Max Preview	100	90	87	84	54	82.9%
Xiaomi MIMO v2.5 Pro	100	100	97	72	41	81.8%
Z.AI GLM 5 Turbo	100	100	100	67	40	81.5%
Qwen 3.5 Plus (2026-04-20)	100	99	86	76	45	81.4%
Gemini 2.5 Pro	100	100	100	70	33	80.7%
GPT-5.2	97	90	83	74	60	80.5%
Qwen 3.6 35B	100	100	88	64	48	80.2%
Nemotron 3 Super	100	100	100	100	0	80.0%
Mistral Small 4	100	100	100	100	0	80.0%
Mistral NeMO	100	100	100	100	0	80.0%
Ministral 3 8B	100	100	83	69	45	79.6%
Arcee AI: Trinity Mini	100	95	83	78	38	78.9%
Gemma 4 26B	100	100	84	75	34	78.5%
Gemma 4 31B	100	93	82	72	43	78.1%
Hermes 3 70B	100	100	100	90	0	78.0%
Qwen 3.6 27B	100	100	96	81	13	78.0%
Aion 3.0	100	100	86	68	33	77.5%
o4 Mini	100	88	73	65	59	77.1%
Gemini 3.1 Flash Lite (Preview)	100	88	76	68	51	76.6%
Gemini 3.5 Flash (Reasoning)	100	100	87	66	29	76.3%
GPT-4.1	100	90	86	69	34	75.9%
ByteDance Seed 1.6 Flash	100	100	83	61	31	75.0%
ByteDance Seed 2.0 Mini	100	100	60	57	56	74.5%
DeepSeek V3 (2025-03-24)	100	100	100	67	0	73.3%
MiniMax M2.5	100	100	94	38	34	73.2%
Gemini 2.5 Flash (Reasoning)	100	100	100	65	0	72.9%
MiniMax M3	100	100	100	37	26	72.8%
Grok 4.3	100	100	85	74	0	72.0%
Z.AI GLM 5.2 (Reasoning, High)	100	100	96	62	0	71.6%
Z.AI GLM 5.1	100	100	100	56	0	71.3%
Qwen 3.5 Plus (2026-02-15)	100	85	77	56	37	71.2%
Cohere Command R+ (Aug. 2024)	100	100	57	53	45	71.1%
Ministral 3B	100	100	85	69	0	71.0%
Gemini 3.1 Pro (Preview)	100	87	71	68	28	70.8%
DeepSeek V4 Flash	100	78	73	48	46	69.0%
Qwen 3.5 397B A17B	100	87	66	46	44	68.7%
Z.AI GLM 4.7 Flash	100	87	69	47	32	66.9%
GPT-5	100	71	65	50	47	66.5%
Cydonia 24B V4.1	100	100	75	56	0	66.2%
Claude Opus 4.6 (Reasoning)	100	100	100	29	0	65.7%
Qwen 3 32B	100	85	69	51	18	64.7%
Claude Opus 4.6	100	94	82	46	0	64.4%
Qwen 3.5 35B	98	80	59	52	32	64.2%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	64	50	0	62.8%
Claude Sonnet 4.6 (Reasoning)	100	100	76	37	0	62.5%
DeepSeek V3.1	100	100	79	29	0	61.5%
Gemma 4 26B (Reasoning)	100	100	38	36	28	60.5%
o4 Mini High	100	100	77	18	0	59.0%
Gemma 4 31B (Reasoning)	100	78	72	40	0	58.0%
Hermes 3 405B	100	100	88	0	0	57.5%
GPT-OSS 120B	77	72	71	55	0	54.8%
Xiaomi MIMO v2.5	81	65	54	46	21	53.5%
GPT-4o Mini (temp=0)	100	81	43	41	0	53.0%
Qwen 3.6 Flash	100	66	49	26	21	52.3%
MoonshotAI: Kimi K2.5	100	77	41	37	0	51.0%
Gemini 3 Flash (Preview)	100	62	60	28	0	50.1%
Gemini 3.1 Flash Lite	100	55	43	38	0	47.1%
Qwen 3.5 122B	100	59	43	33	0	47.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	29	0	0	45.7%
Qwen 3.5 Flash	79	75	23	22	0	40.0%
GPT-4o, Aug. 6th (temp=0)	93	56	42	5	0	39.0%
GPT-5 Mini	50	48	39	24	22	36.6%
ByteDance Seed 1.6	62	61	60	0	0	36.4%
Qwen3.7 Max	70	57	49	0	0	35.3%
Qwen 2.5 72B	100	61	0	0	0	32.2%
Llama 3.1 70B	88	48	17	0	0	30.5%
Gemini 3.1 Flash Lite (Reasoning)	73	69	0	0	0	28.5%
Grok 4.3 (Reasoning)	100	32	0	0	0	26.4%
Qwen 3.5 9B	47	29	26	17	0	23.9%
Qwen 3.5 27B	71	24	15	0	0	22.0%
Inception Mercury 2	42	41	0	0	0	16.7%
Mistral Small 3.2 24B	65	12	4	1	0	16.4%
ByteDance Seed 2.0 Lite	51	0	0	0	0	10.3%
Nemotron 3 Nano	33	0	0	0	0	6.5%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	98	99.6%
Mistral Small 4	100	100	100	100	93	98.6%
Mistral Large 3	100	100	100	100	92	98.3%
Mistral Large 2	100	100	100	100	85	97.1%
Claude Haiku 4.5	100	100	100	83	82	93.1%
Claude Sonnet 4.6 (Reasoning)	100	100	100	83	81	92.9%
Mistral Small 4 (Reasoning)	100	100	100	100	60	91.9%
Cydonia 24B V4.1	100	100	100	97	53	89.9%
Ministral 8B	100	100	100	100	48	89.7%
Claude Opus 4.7	100	100	100	100	48	89.7%
Gemma 3 27B	100	100	98	77	72	89.4%
Qwen 3 32B	100	100	100	100	44	88.8%
DeepSeek V4 Pro (Reasoning)	100	100	100	94	40	86.8%
GPT-5 Nano	100	100	100	76	57	86.6%
Ministral 3 14B	100	100	98	85	43	85.2%
MiniMax M2.7	100	100	91	70	43	80.8%
Gemma 3 4B	100	100	100	100	0	80.0%
GPT-4.1	100	89	72	70	54	77.1%
GPT-5.4 Nano	100	98	67	63	57	76.9%
Claude Opus 4	100	100	80	75	26	76.3%
Mistral Medium 3.1	100	100	71	63	46	76.0%
GPT-5.4	100	72	70	70	66	75.6%
Claude Opus 4.7 (Reasoning)	100	79	72	69	55	75.2%
MiniMax M2.5	100	100	81	51	40	74.5%
Xiaomi MIMO v2.5 Pro	100	100	98	38	36	74.4%
Claude Opus 4.8 (Reasoning, Low)	100	76	69	68	56	73.9%
GPT-5.5	100	75	71	65	57	73.6%
Claude Opus 4.8 (Reasoning)	100	100	100	65	0	73.0%
Hermes 3 70B	100	100	100	58	0	71.7%
Ministral 3B	100	100	100	58	0	71.7%
DeepSeek V3.2	100	100	65	59	34	71.6%
DeepSeek-V2 Chat	100	100	100	52	0	70.4%
GPT-5.5 (Reasoning, Low)	100	95	53	53	49	69.9%
GPT-4.1 Nano	100	100	100	49	0	69.8%
GPT-5.4 Nano (Reasoning)	100	100	75	37	33	68.9%
GPT-5.2	100	78	61	57	41	67.3%
o4 Mini High	100	73	62	57	42	66.9%
Z.AI GLM 5	100	73	66	56	37	66.3%
GPT-5.4 (Reasoning, Low)	100	100	60	48	23	66.1%
Claude Sonnet 4.5	100	100	61	35	31	65.3%
Claude Opus 4.5	100	97	88	40	0	64.8%
Gemini 2.5 Flash (Reasoning)	100	100	84	38	0	64.5%
Qwen 3.5 397B A17B	92	81	73	49	24	63.9%
GPT-5.4 Nano (Reasoning, Low)	84	74	73	45	41	63.6%
o4 Mini	100	68	63	56	30	63.5%
Grok 4.20 (Reasoning)	100	100	47	43	27	63.5%
GPT-5.4 Mini (Reasoning)	100	88	52	47	28	62.9%
GPT-5.5 (Reasoning)	100	79	67	38	30	62.9%
Claude Opus 4.6	100	93	83	30	0	61.2%
Ministral 3 3B	100	100	51	42	1	58.9%
Gemini 2.5 Flash Lite (Reasoning)	99	81	62	51	0	58.6%
Z.AI GLM 5.2 (Reasoning, High)	100	97	53	39	0	57.9%
GPT-5	100	100	31	29	28	57.7%
Claude Sonnet 4.6	100	53	48	44	43	57.7%
Z.AI GLM 5.1	100	77	73	37	0	57.4%
Claude Sonnet 5	100	100	81	0	0	56.3%
DeepSeek V4 Flash	100	100	81	0	0	56.3%
GPT-5.4 Mini	100	100	53	26	0	55.9%
DeepSeek V4 Pro	100	100	40	40	0	55.9%
DeepSeek V3 (2025-03-24)	100	100	74	0	0	54.8%
Claude Sonnet 4	100	88	43	43	0	54.7%
ByteDance Seed 2.0 Lite	100	88	71	0	0	51.7%
Claude Sonnet 5 (Reasoning, Low)	100	90	68	0	0	51.6%
WizardLM 2 8x22b	100	93	32	32	0	51.3%
Aion 3.0 Mini	100	67	55	35	0	51.2%
Z.AI GLM 4.7	72	68	42	35	32	50.0%
GPT-5 Mini	68	68	45	41	25	49.5%
GPT-5.4 (Reasoning)	100	81	42	23	0	49.3%
Gemma 3 12B	100	100	46	0	0	49.3%
Grok 4.20	74	68	61	40	0	48.7%
Grok 4.5 (Reasoning, Low)	100	52	50	41	0	48.6%
MiniMax M3	82	78	46	34	0	48.0%
ByteDance Seed 1.6 Flash	100	78	32	24	0	47.0%
Cohere Command R+ (Aug. 2024)	90	83	61	0	0	46.8%
Gemini 2.5 Pro	100	84	44	0	0	45.7%
Z.AI GLM 4.6	77	62	55	35	0	45.6%
Aion 3.0	100	89	37	0	0	45.2%
Qwen 3.5 Plus (2026-04-20)	72	52	41	34	27	45.0%
DeepSeek V4 Flash (Reasoning)	100	80	39	0	0	43.8%
GPT-4o, Aug. 6th (temp=1)	100	64	53	0	0	43.4%
GPT-5.1	81	72	30	29	0	42.3%
ByteDance Seed 2.0 Mini	100	57	53	0	0	42.1%
GPT-5.4 Mini (Reasoning, Low)	74	47	28	26	25	40.0%
Llama 3.1 70B	100	100	0	0	0	40.0%
Qwen 3.5 Flash	75	53	45	25	0	39.7%
Aion 2.0	85	72	35	0	0	38.5%
GPT-4o Mini (temp=1)	100	93	0	0	0	38.5%
Gemini 2.5 Flash	100	49	37	0	0	37.3%
Z.AI GLM 5 Turbo	83	56	46	0	0	37.1%
Qwen 3.5 122B	100	36	29	19	0	36.9%
Claude Sonnet 5 (Reasoning)	93	88	0	0	0	36.1%
Xiaomi MIMO v2.5	63	44	39	30	0	35.2%
ByteDance Seed 1.6	95	72	0	0	0	33.5%
Gemini 3.1 Flash Lite	100	62	0	0	0	32.3%
Qwen3.6 Max Preview	94	23	21	21	0	31.7%
DeepSeek V3.1	97	31	31	0	0	31.6%
Qwen 3.6 Flash	100	39	19	0	0	31.6%
Mistral NeMO	100	56	0	0	0	31.1%
Hermes 3 405B	100	55	0	0	0	31.0%
MoonshotAI: Kimi K2.5	89	63	0	0	0	30.4%
Z.AI GLM 4.5 Air	100	47	0	0	0	29.4%
Qwen 3.5 27B	81	35	27	0	0	28.6%
Grok 4.5 (Reasoning, High)	93	46	0	0	0	27.7%
Gemini 2.5 Flash Lite	95	42	0	0	0	27.4%
GPT-4.1 Mini	76	61	0	0	0	27.3%
MoonshotAI: Kimi K2.6	53	45	34	0	0	26.5%
Gemma 4 31B (Reasoning)	62	55	0	0	0	23.3%
Claude Opus 4.6 (Reasoning)	68	45	0	0	0	22.6%
Gemini 3.1 Flash Lite (Preview)	56	53	0	0	0	21.9%
Qwen 3.5 Plus (2026-02-15)	76	33	0	0	0	21.8%
Gemini 3 Flash (Preview)	38	35	34	0	0	21.3%
Nemotron 3 Super	69	36	0	0	0	21.1%
Z.AI GLM 4.7 Flash	41	31	30	0	0	20.5%
Qwen 3.5 35B	49	23	21	8	0	20.4%
Qwen 3.6 27B	45	26	25	0	0	19.2%
Gemma 4 26B	48	44	0	0	0	18.6%
Nemotron 3 Nano	54	28	0	0	0	16.3%
Qwen 3.6 35B	39	30	5	0	0	14.7%
Qwen 3.5 9B	33	29	6	0	0	13.7%
Gemini 3.1 Pro (Preview)	35	32	0	0	0	13.5%
Qwen 2.5 72B	53	0	0	0	0	10.6%
Mistral Small 3.2 24B	50	1	1	0	0	10.5%
Gemma 4 31B	49	0	0	0	0	9.8%
DeepSeek V3 (2024-12-26)	48	0	0	0	0	9.5%
Z.AI GLM 4.5	46	0	0	0	0	9.3%
Qwen3.7 Max	46	0	0	0	0	9.1%
Gemma 4 26B (Reasoning)	45	0	0	0	0	9.0%
Arcee AI: Trinity Mini	42	0	0	0	0	8.4%
GPT-OSS 120B	40	0	0	0	0	8.0%
GPT-4o Mini (temp=0)	36	0	0	0	0	7.2%
Gemini 3 Flash (Preview, Reasoning)	34	0	0	0	0	6.9%
Gemini 3.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Grok 4.3 (Reasoning)	0	0	0	0	0	0.0%
Gemini 3.1 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Gemini 3.5 Flash (Reasoning, Minimal)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Grok 4.3	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	100	100.0%
Z.AI GLM 5.2 (Reasoning, High)	100	100	100	100	100	99.9%
Qwen3 235B A22B Instruct 2507	100	100	100	100	91	98.3%
MiniMax M3	100	100	100	94	62	91.1%
Mistral Large 3	100	100	91	86	74	90.3%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	85	62	89.4%
GPT-5.4 (Reasoning, Low)	100	100	100	98	42	88.1%
Claude Sonnet 4.5	100	100	100	100	39	87.8%
Mistral Medium 3.1	100	100	100	97	27	84.8%
Claude Opus 4.8 (Reasoning)	100	100	83	80	49	82.5%
Mistral Small 4 (Reasoning)	100	93	82	81	56	82.4%
Cydonia 24B V4.1	100	100	100	100	0	80.0%
Ministral 3 3B	100	100	100	93	0	78.5%
Ministral 3 14B	100	100	100	91	0	78.3%
GPT-5.4	100	100	74	66	42	76.6%
Gemma 4 26B (Reasoning)	100	97	97	87	0	76.0%
Ministral 3B	100	100	100	65	0	73.1%
Claude Opus 4	100	93	70	49	40	70.6%
Qwen 3.6 35B	100	95	64	47	43	69.8%
GPT-5.4 Mini (Reasoning, Low)	100	76	75	70	27	69.8%
MiniMax M2.5	100	100	100	48	0	69.5%
GPT-5.4 Nano	78	77	69	67	55	69.2%
Aion 3.0 Mini	100	100	100	46	0	69.1%
Claude Opus 4.7 (Reasoning)	100	100	100	43	0	68.7%
Gemini 2.5 Flash Lite	100	100	78	63	0	68.3%
Gemini 2.5 Flash	100	100	94	43	0	67.3%
MiniMax M2.7	100	100	100	37	0	67.3%
Z.AI GLM 4.7	100	69	68	65	33	67.1%
GPT-5.4 Nano (Reasoning, Low)	75	74	71	67	47	66.8%
Ministral 3 8B	100	100	91	40	0	66.3%
Z.AI GLM 5.1	100	100	91	38	0	65.8%
Qwen 3 32B	100	72	61	60	35	65.3%
GPT-5.4 Mini (Reasoning)	100	86	66	47	26	65.1%
Gemma 3 4B	100	95	83	46	0	65.0%
Z.AI GLM 5	88	82	76	41	34	64.1%
Mistral Small 4	100	100	54	34	31	63.8%
Qwen 3.5 397B A17B	100	100	81	36	0	63.3%
GPT-5.1	97	74	60	43	41	62.9%
GPT-5.4 Mini	100	66	64	60	24	62.9%
Mistral Large 2	100	100	74	40	0	62.8%
Grok 4.5 (Reasoning, Low)	100	90	63	60	0	62.5%
Gemini 2.5 Flash Lite (Reasoning)	100	90	63	59	0	62.4%
GPT-5.4 (Reasoning)	100	100	63	46	0	61.8%
Gemma 3 27B	100	82	43	43	40	61.7%
GPT-5.5 (Reasoning)	89	74	65	41	37	60.9%
Claude Sonnet 5 (Reasoning)	100	100	100	0	0	60.0%
Ministral 8B	100	100	100	0	0	60.0%
Gemini 3.1 Flash Lite (Preview)	100	100	51	40	0	58.3%
Qwen 3.6 Flash	97	95	83	16	0	58.1%
Grok 4.5 (Reasoning, High)	100	100	89	0	0	57.8%
Z.AI GLM 4.5 Air	100	100	89	0	0	57.8%
Claude Sonnet 4.6 (Reasoning)	87	80	45	39	37	57.6%
DeepSeek V3 (2025-03-24)	100	100	88	0	0	57.5%
GPT-5.4 Nano (Reasoning)	93	78	46	45	25	57.3%
Claude Opus 4.6 (Reasoning)	100	66	61	60	0	57.2%
Z.AI GLM 4.7 Flash	100	80	78	24	0	56.6%
Qwen3.6 Max Preview	100	90	46	24	21	56.2%
Aion 3.0	100	100	39	39	0	55.7%
Gemma 4 26B	100	72	67	39	0	55.6%
Hermes 3 70B	100	78	55	42	0	54.8%
Z.AI GLM 5 Turbo	100	94	44	35	0	54.8%
DeepSeek V4 Pro (Reasoning)	100	72	52	49	0	54.7%
Grok 4.20	100	81	63	26	0	54.0%
DeepSeek V3.2	100	84	78	0	0	52.3%
GPT-5.5 (Reasoning, Low)	100	71	51	38	0	52.0%
Grok 4.20 (Reasoning)	100	100	30	28	0	51.7%
o4 Mini	100	68	34	33	22	51.5%
GPT-5.2	91	73	35	34	20	50.6%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	48	0	0	49.7%
DeepSeek V4 Flash	100	58	54	34	0	49.3%
GPT-5 Nano	88	60	44	44	0	47.1%
Gemini 3 Flash (Preview, Reasoning)	72	69	54	40	0	47.0%
o4 Mini High	100	73	43	19	0	46.9%
Gemma 3 12B	100	83	50	0	0	46.6%
MoonshotAI: Kimi K2.6	95	65	40	33	0	46.6%
Claude Opus 4.5	95	91	46	0	0	46.6%
Qwen 3.5 Plus (2026-02-15)	68	60	52	31	22	46.4%
Gemini 3.1 Flash Lite	79	56	50	44	0	45.8%
Gemini 3.5 Flash (Reasoning)	100	78	51	0	0	45.6%
Grok 4.3	100	45	45	37	0	45.5%
DeepSeek V3.1	100	89	35	0	0	44.8%
Qwen 3.5 122B	97	85	39	0	0	44.2%
Arcee AI: Trinity Mini	100	65	55	0	0	44.0%
DeepSeek V4 Pro	80	52	43	41	0	43.3%
WizardLM 2 8x22b	97	58	33	23	0	42.3%
Qwen 3.5 27B	88	71	25	23	0	41.8%
Xiaomi MIMO v2.5	65	57	35	27	23	41.5%
GPT-5.5	100	54	36	16	0	41.3%
Claude Opus 4.6	100	62	43	0	0	41.1%
Claude Sonnet 4.6	100	52	48	0	0	40.1%
Gemma 4 31B	100	100	0	0	0	40.0%
Gemini 3 Flash (Preview)	89	73	37	0	0	39.8%
Qwen 3.5 Plus (2026-04-20)	62	55	52	28	0	39.4%
Claude Opus 4.8 (Reasoning, Low)	100	50	46	0	0	39.1%
GPT-4o, Aug. 6th (temp=0)	76	58	56	0	0	38.0%
Gemini 3.1 Flash Lite (Reasoning)	100	89	0	0	0	37.7%
Gemini 2.5 Flash (Reasoning)	78	63	44	0	0	37.1%
Claude Sonnet 5	100	79	0	0	0	35.9%
DeepSeek V4 Flash (Reasoning)	100	38	36	0	0	34.8%
Claude Haiku 4.5	73	64	30	0	0	33.5%
Qwen 3.6 27B	100	33	32	0	0	33.1%
Z.AI GLM 4.5	93	72	0	0	0	33.0%
GPT-5	50	46	37	14	14	32.1%
MoonshotAI: Kimi K2.5	100	58	0	0	0	31.7%
Xiaomi MIMO v2.5 Pro	100	54	0	0	0	30.8%
GPT-4.1 Nano	100	54	0	0	0	30.8%
GPT-4o Mini (temp=1)	100	50	0	0	0	30.0%
GPT-4o, Aug. 6th (temp=1)	74	69	0	0	0	28.7%
ByteDance Seed 1.6 Flash	100	42	0	0	0	28.3%
Gemma 4 31B (Reasoning)	54	46	42	0	0	28.2%
Gemini 2.5 Pro	100	40	0	0	0	28.0%
DeepSeek V3 (2024-12-26)	100	34	0	0	0	26.7%
GPT-5 Mini	94	37	0	0	0	26.2%
Z.AI GLM 4.6	47	38	37	0	0	24.5%
Nemotron 3 Super	60	59	0	0	0	23.7%
Qwen 3.5 35B	40	35	17	12	11	23.2%
Claude Sonnet 4	68	42	0	0	0	22.0%
Claude Opus 4.7	57	51	0	0	0	21.8%
GPT-4.1	100	0	0	0	0	20.0%
ByteDance Seed 2.0 Mini	100	0	0	0	0	20.0%
Hermes 3 405B	100	0	0	0	0	20.0%
Qwen 2.5 72B	100	0	0	0	0	20.0%
Cohere Command R+ (Aug. 2024)	100	0	0	0	0	20.0%
Qwen3.7 Max	49	47	0	0	0	19.2%
ByteDance Seed 2.0 Lite	95	0	0	0	0	19.0%
Qwen 3.5 Flash	93	0	0	0	0	18.7%
Grok 4.3 (Reasoning)	43	40	0	0	0	16.6%
Aion 2.0	43	35	0	0	0	15.8%
Gemini 3.1 Pro (Preview)	38	36	0	0	0	14.8%
Qwen 3.5 9B	44	10	0	0	0	10.9%
GPT-4.1 Mini	46	0	0	0	0	9.3%
GPT-4o Mini (temp=0)	44	0	0	0	0	8.8%
Mistral NeMO	41	0	0	0	0	8.2%
DeepSeek-V2 Chat	40	0	0	0	0	7.9%
Mistral Small 3.2 24B	1	0	0	0	0	0.2%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	93	90	96.5%
Gemma 3 12B	100	100	100	100	82	96.5%
Qwen3 235B A22B Instruct 2507	100	100	100	90	76	93.2%
Claude Opus 4	100	100	100	100	65	93.0%
GPT-5.4 (Reasoning, Low)	100	100	100	85	80	93.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	95	89	72	91.4%
Ministral 8B	100	100	100	82	72	90.7%
GPT-5.4 Mini	100	100	100	97	57	90.7%
Mistral Medium 3.1	100	100	100	79	66	89.1%
Grok 4.20	100	100	100	79	65	88.7%
GPT-5 Nano	100	100	100	75	66	88.2%
Claude Sonnet 4.6 (Reasoning)	100	100	83	81	76	88.1%
GPT-5.5	100	100	80	78	77	87.0%
Mistral Large 2	100	100	100	71	46	83.3%
Gemma 3 4B	100	100	100	56	52	81.7%
GPT-5.4 Mini (Reasoning)	100	100	100	82	23	80.9%
GPT-5.5 (Reasoning)	100	99	99	78	30	80.9%
MiniMax M2.7	100	100	100	99	0	79.8%
Z.AI GLM 5.2 (Reasoning, High)	100	96	77	72	52	79.5%
GPT-5.4 Nano	100	95	87	69	46	79.3%
Claude Opus 4.6 (Reasoning)	100	100	96	68	31	78.9%
GPT-5.5 (Reasoning, Low)	100	100	85	71	34	77.9%
GPT-5.4	100	100	100	47	43	77.9%
Mistral Small 4 (Reasoning)	100	100	100	89	0	77.9%
Grok 4.5 (Reasoning, Low)	100	100	100	48	37	77.2%
Gemini 2.5 Flash Lite	100	91	73	70	48	76.6%
Xiaomi MIMO v2.5 Pro	100	100	100	83	0	76.5%
Ministral 3 14B	100	100	64	63	50	75.3%
Gemma 3 27B	100	100	100	74	0	74.8%
Hermes 3 70B	100	100	100	74	0	74.7%
MiniMax M3	100	77	73	68	49	73.4%
Claude Opus 4.8 (Reasoning, Low)	100	98	58	55	55	73.2%
MiniMax M2.5	100	100	91	74	0	73.1%
Claude Opus 4.6	100	88	72	68	36	72.7%
Aion 3.0	91	73	69	68	58	71.9%
DeepSeek V4 Flash (Reasoning)	100	100	100	58	0	71.7%
Claude Sonnet 5 (Reasoning, Low)	100	100	81	74	0	71.1%
GPT-5.4 Nano (Reasoning)	87	84	69	60	55	70.8%
Mistral Large 3	100	100	100	48	0	69.5%
Grok 4.20 (Reasoning)	100	88	81	49	23	68.2%
WizardLM 2 8x22b	100	100	64	35	30	66.0%
Ministral 3 3B	100	100	74	38	17	65.8%
GPT-4.1 Nano	100	100	65	62	0	65.4%
Z.AI GLM 5	100	100	80	46	0	65.3%
GPT-4o, Aug. 6th (temp=1)	100	100	64	61	0	64.9%
Claude Sonnet 4	100	97	93	32	0	64.4%
Ministral 3 8B	100	100	100	14	0	62.7%
Z.AI GLM 5 Turbo	100	67	54	51	42	62.7%
GPT-5.4 (Reasoning)	97	93	57	45	21	62.6%
Gemini 3.1 Flash Lite	98	93	64	56	0	62.1%
Gemma 4 26B	100	100	60	51	0	62.0%
GPT-5 Mini	100	83	70	28	26	61.6%
ByteDance Seed 1.6 Flash	100	76	61	37	31	61.0%
GPT-4.1 Mini	100	100	65	38	0	60.4%
o4 Mini	79	71	69	53	28	60.1%
GPT-5.1	95	85	69	25	22	59.1%
Ministral 3B	100	100	79	16	0	58.9%
Claude Opus 4.5	100	100	50	44	0	58.8%
Claude Haiku 4.5	100	100	89	0	0	57.7%
Cydonia 24B V4.1	100	80	57	50	0	57.6%
Hermes 3 405B	100	69	63	55	0	57.4%
Claude Sonnet 5 (Reasoning)	100	100	85	0	0	57.1%
Claude Sonnet 4.5	100	100	41	39	0	56.0%
Qwen 3 32B	100	98	67	0	0	52.9%
Gemini 3 Flash (Preview, Reasoning)	97	83	40	39	0	51.7%
Z.AI GLM 4.5 Air	100	100	56	0	0	51.3%
Qwen 3.5 Flash	81	53	51	48	18	50.3%
Qwen 3.6 Flash	82	69	41	34	20	49.4%
Gemini 2.5 Pro	80	72	47	44	0	48.8%
DeepSeek V4 Pro	100	76	35	32	0	48.7%
DeepSeek V4 Flash	100	100	40	0	0	47.9%
Claude Opus 4.8 (Reasoning)	71	60	55	52	0	47.4%
Z.AI GLM 4.7 Flash	78	72	55	31	0	47.3%
Gemini 2.5 Flash Lite (Reasoning)	100	44	33	30	29	47.2%
o4 Mini High	65	63	52	28	27	47.2%
MoonshotAI: Kimi K2.5	100	93	40	0	0	46.6%
ByteDance Seed 2.0 Mini	100	76	54	0	0	45.9%
Gemini 3.1 Flash Lite (Preview)	100	74	53	0	0	45.4%
Gemini 3.5 Flash (Reasoning, Minimal)	100	64	58	0	0	44.5%
DeepSeek V4 Pro (Reasoning)	100	62	56	0	0	43.6%
Qwen 3.5 397B A17B	100	53	42	19	0	42.8%
Claude Opus 4.7 (Reasoning)	100	56	56	0	0	42.4%
Z.AI GLM 5.1	90	83	38	0	0	42.3%
Qwen 3.6 35B	100	52	34	22	0	41.4%
Qwen 3.5 Plus (2026-02-15)	100	52	52	0	0	40.8%
Claude Opus 4.7	100	54	48	0	0	40.4%
Xiaomi MIMO v2.5	100	64	36	0	0	40.1%
Claude Sonnet 4.6	100	100	0	0	0	40.0%
Mistral NeMO	100	100	0	0	0	40.0%
DeepSeek V3 (2025-03-24)	100	61	39	0	0	40.0%
Aion 3.0 Mini	100	58	39	0	0	39.3%
Gemma 4 26B (Reasoning)	98	54	44	0	0	39.1%
Aion 2.0	100	41	29	26	0	39.0%
Gemini 2.5 Flash (Reasoning)	90	65	35	0	0	38.0%
MoonshotAI: Kimi K2.6	69	48	40	33	0	37.8%
Claude Sonnet 5	100	81	0	0	0	36.3%
Gemini 2.5 Flash	94	48	39	0	0	36.1%
DeepSeek V3.1	95	45	38	0	0	35.7%
GPT-4.1	88	45	42	0	0	35.0%
Gemini 3 Flash (Preview)	50	47	40	31	0	33.4%
Arcee AI: Trinity Mini	63	51	48	0	0	32.5%
Qwen 3.5 Plus (2026-04-20)	100	40	21	0	0	32.2%
Grok 4.3 (Reasoning)	65	58	35	0	0	31.9%
Gemma 4 31B	100	56	0	0	0	31.3%
Grok 4.3	60	52	43	0	0	31.0%
Grok 4.5 (Reasoning, High)	78	38	37	0	0	30.8%
GPT-4o Mini (temp=1)	100	54	0	0	0	30.8%
Mistral Small 4	95	54	0	0	0	29.8%
Z.AI GLM 4.7	64	45	39	0	0	29.6%
Z.AI GLM 4.6	100	48	0	0	0	29.5%
ByteDance Seed 1.6	78	67	0	0	0	28.8%
Qwen3.6 Max Preview	40	36	31	31	0	27.7%
Nemotron 3 Super	49	42	32	0	0	24.7%
Qwen 3.5 122B	56	43	22	0	0	24.2%
Cohere Command R+ (Aug. 2024)	93	28	0	0	0	24.2%
Gemma 4 31B (Reasoning)	69	51	0	0	0	24.1%
Gemini 3.5 Flash (Reasoning)	48	37	33	0	0	23.4%
DeepSeek-V2 Chat	72	42	0	0	0	22.9%
Nemotron 3 Nano	47	44	20	0	0	22.3%
Qwen 3.6 27B	31	22	20	17	16	21.2%
Qwen 3.5 27B	29	27	24	22	0	20.4%
ByteDance Seed 2.0 Lite	100	0	0	0	0	20.0%
GPT-4o, Aug. 6th (temp=0)	100	0	0	0	0	20.0%
GPT-5	59	21	19	0	0	19.9%
Mistral Small 3.2 24B	88	0	0	0	0	17.7%
DeepSeek V3.2	39	32	0	0	0	14.2%
Qwen 3.5 9B	69	0	0	0	0	13.9%
Qwen 3.5 35B	33	24	12	0	0	13.8%
Qwen3.7 Max	66	0	0	0	0	13.2%
GPT-5.2	48	18	0	0	0	13.0%
DeepSeek V3 (2024-12-26)	58	0	0	0	0	11.7%
Qwen 2.5 72B	48	0	0	0	0	9.7%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
Gemini 3.1 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
Z.AI GLM 4.5	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 5	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	89	97.8%
Mistral Medium 3.1	100	100	100	100	87	97.3%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	85	97.1%
GPT-5.4 Nano	100	100	100	100	85	96.9%
Ministral 3 14B	100	100	100	100	84	96.7%
GPT-5.5 (Reasoning, Low)	100	100	100	100	82	96.5%
Ministral 3B	100	100	98	95	85	95.7%
GPT-5.4 (Reasoning, Low)	100	100	95	90	73	91.6%
GPT-5.4 Mini	100	99	90	88	77	90.7%
Z.AI GLM 5 Turbo	100	100	100	82	69	90.3%
GPT-5.4	100	100	100	71	70	88.2%
Mistral Large 3	100	100	100	100	38	87.6%
Hermes 3 70B	100	100	100	69	64	86.7%
Mistral Small 4	100	100	100	100	32	86.4%
GPT-5 Nano	100	89	81	81	76	85.3%
GPT-5.5	100	100	100	81	37	83.5%
Qwen 3.5 397B A17B	100	95	91	71	55	82.4%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	61	51	82.4%
MiniMax M3	100	100	100	77	32	81.7%
GPT-5.4 Nano (Reasoning, Low)	100	100	79	78	47	80.7%
Mistral Large 2	100	100	100	57	43	80.2%
Claude Opus 4.7	100	100	100	100	0	80.0%
Mistral Small 4 (Reasoning)	100	100	100	100	0	80.0%
Z.AI GLM 5.2 (Reasoning, High)	100	100	84	79	28	78.4%
GPT-5.5 (Reasoning)	100	95	84	70	42	78.1%
MiniMax M2.7	100	100	78	56	52	77.0%
Xiaomi MIMO v2.5 Pro	100	100	98	62	22	76.3%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	79	0	75.9%
GPT-5.4 (Reasoning)	90	76	74	74	65	75.8%
Claude Sonnet 5 (Reasoning)	100	100	56	56	56	73.9%
Gemini 2.5 Flash	100	98	83	53	33	73.5%
Aion 3.0 Mini	100	83	67	64	49	72.8%
GPT-5.4 Mini (Reasoning, Low)	100	86	74	57	42	71.7%
Ministral 3 8B	100	100	89	67	0	71.4%
Gemma 3 27B	100	100	85	38	33	71.3%
Z.AI GLM 4.5	100	100	72	44	38	71.0%
GPT-5.1	100	85	74	47	44	70.0%
MiniMax M2.5	100	100	100	50	0	70.0%
Ministral 3 3B	100	100	74	72	0	69.3%
Aion 3.0	100	70	63	54	53	68.0%
Claude Opus 4.7 (Reasoning)	100	100	65	37	36	67.6%
Gemma 3 12B	100	100	97	35	0	66.4%
Gemma 3 4B	100	100	46	45	41	66.4%
DeepSeek V4 Flash (Reasoning)	100	98	93	41	0	66.4%
Z.AI GLM 4.5 Air	100	100	66	34	32	66.3%
Grok 4.20	100	93	47	46	42	65.7%
Claude Opus 4.6 (Reasoning)	84	75	62	54	54	65.5%
Claude Sonnet 4.6	100	100	82	43	0	65.0%
Xiaomi MIMO v2.5	92	78	75	45	32	64.4%
DeepSeek V4 Pro (Reasoning)	100	100	82	37	0	63.9%
Grok 4.5 (Reasoning, High)	100	100	67	52	0	63.8%
GPT-4.1	91	81	60	56	29	63.3%
Mistral NeMO	100	69	58	49	37	62.7%
Z.AI GLM 4.7 Flash	96	89	84	44	0	62.7%
GPT-4.1 Mini	100	88	83	42	0	62.6%
Claude Haiku 4.5	100	80	69	36	27	62.5%
GPT-5.4 Mini (Reasoning)	100	83	50	50	29	62.3%
Claude Opus 4.5	100	69	61	43	37	62.0%
Z.AI GLM 5.1	100	83	81	46	0	61.9%
Claude Opus 4	86	65	63	52	41	61.3%
DeepSeek V4 Flash	82	76	59	56	34	61.3%
Gemma 4 26B	98	81	62	61	0	60.2%
DeepSeek V3 (2025-03-24)	100	100	100	0	0	59.9%
DeepSeek V4 Pro	100	100	57	42	0	59.7%
Claude Opus 4.6	74	65	64	48	45	59.4%
Arcee AI: Trinity Mini	100	71	65	56	0	58.4%
Gemini 3.1 Flash Lite (Reasoning)	100	100	46	45	0	58.3%
DeepSeek-V2 Chat	100	100	91	0	0	58.3%
Gemini 3 Flash (Preview)	100	100	60	31	0	58.1%
Qwen 3.5 Plus (2026-02-15)	100	85	61	44	0	58.0%
Cydonia 24B V4.1	100	98	91	0	0	57.9%
Z.AI GLM 4.6	100	87	38	31	31	57.5%
Gemini 2.5 Pro	100	85	70	31	0	57.4%
Grok 4.20 (Reasoning)	82	79	56	46	23	57.1%
o4 Mini High	100	88	59	38	0	57.0%
Hermes 3 405B	100	100	81	0	0	56.3%
Claude Opus 4.8 (Reasoning)	82	69	44	41	41	55.3%
WizardLM 2 8x22b	100	80	57	38	0	55.0%
DeepSeek V3.2	100	69	52	29	25	54.9%
Grok 4.5 (Reasoning, Low)	100	96	78	0	0	54.9%
Claude Sonnet 4.5	100	56	56	31	29	54.5%
GPT-5.4 Nano (Reasoning)	100	67	57	30	19	54.5%
Ministral 8B	100	89	81	0	0	54.0%
Cohere Command R+ (Aug. 2024)	100	64	61	44	0	53.8%
Z.AI GLM 4.7	88	75	52	33	21	53.8%
Claude Sonnet 4	80	78	73	37	0	53.6%
GPT-5.2	100	62	35	34	31	52.3%
ByteDance Seed 1.6 Flash	88	67	56	25	23	51.9%
GPT-4o, Aug. 6th (temp=1)	100	54	53	50	0	51.3%
o4 Mini	75	58	45	43	35	51.1%
Aion 2.0	100	56	56	34	0	49.1%
MoonshotAI: Kimi K2.5	74	49	48	37	35	48.6%
Qwen 3.6 Flash	100	71	31	26	13	47.9%
Gemini 3.5 Flash (Reasoning)	81	68	46	43	0	47.8%
DeepSeek V3 (2024-12-26)	100	100	36	0	0	47.2%
MoonshotAI: Kimi K2.6	100	100	29	0	0	45.8%
Qwen 3.5 Plus (2026-04-20)	86	77	32	30	0	45.0%
GPT-4o Mini (temp=1)	91	82	49	0	0	44.5%
Qwen 2.5 72B	83	61	35	34	0	42.7%
Gemini 3.5 Flash (Reasoning, Minimal)	100	69	44	0	0	42.7%
Gemini 2.5 Flash Lite (Reasoning)	100	82	29	0	0	42.4%
Gemini 3 Flash (Preview, Reasoning)	100	79	23	0	0	40.5%
Nemotron 3 Super	81	71	48	0	0	40.0%
Gemini 2.5 Flash Lite	80	40	39	37	0	39.2%
Qwen 3 32B	81	55	38	16	0	38.0%
DeepSeek V3.1	80	49	28	23	0	36.1%
Gemma 4 26B (Reasoning)	40	38	35	33	32	35.7%
Qwen3.6 Max Preview	87	51	34	0	0	34.5%
GPT-4o Mini (temp=0)	47	40	33	33	0	30.6%
GPT-4o, Aug. 6th (temp=0)	100	52	0	0	0	30.4%
Qwen 3.6 27B	92	59	0	0	0	30.3%
ByteDance Seed 1.6	76	68	0	0	0	28.8%
Gemini 3.1 Flash Lite	98	44	0	0	0	28.4%
Qwen 3.5 27B	43	40	35	14	0	26.4%
GPT-5	49	27	27	14	9	25.3%
Gemini 2.5 Flash (Reasoning)	57	55	0	0	0	22.5%
Qwen 3.6 35B	36	31	30	14	0	22.3%
Qwen 3.5 35B	42	34	19	17	0	22.2%
Qwen 3.5 9B	43	28	24	0	0	19.1%
Grok 4.3	60	32	0	0	0	18.4%
Qwen 3.5 122B	37	24	19	10	0	18.3%
Grok 4.3 (Reasoning)	45	39	0	0	0	16.8%
Gemma 4 31B (Reasoning)	45	39	0	0	0	16.8%
Qwen 3.5 Flash	42	39	0	0	0	16.3%
Mistral Small 3.2 24B	79	1	0	0	0	16.1%
ByteDance Seed 2.0 Mini	74	0	0	0	0	14.8%
Llama 3.1 70B	67	0	0	0	0	13.3%
Nemotron 3 Nano	48	0	0	0	0	9.6%
Gemini 3.1 Flash Lite (Preview)	45	0	0	0	0	9.0%
Gemma 4 31B	37	0	0	0	0	7.4%
Qwen3.7 Max	35	0	0	0	0	7.1%
Gemini 3.1 Pro (Preview)	27	0	0	0	0	5.4%
GPT-5 Mini	19	0	0	0	0	3.9%
GPT-OSS 120B	0	0	0	0	0	0.0%
ByteDance Seed 2.0 Lite	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Grok 4.5 (Reasoning, High)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.5 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	99	99.7%
GPT-5.5	100	100	100	100	96	99.1%
GPT-5.4 Mini (Reasoning)	100	100	100	100	93	98.5%
GPT-5.4 (Reasoning, Low)	100	100	100	100	92	98.4%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	81	96.2%
Mistral Medium 3.1	100	100	100	84	82	93.3%
Grok 4.20 (Reasoning)	100	100	98	90	76	92.8%
Claude Opus 4.7	100	100	100	100	60	91.9%
Mistral Large 2	100	100	91	89	68	89.6%
GPT-5.2	100	100	100	93	53	89.2%
Writer: Palmyra X5	100	100	100	100	43	88.7%
GPT-5.4 Nano	100	100	83	78	77	87.5%
GPT-5.5 (Reasoning)	100	100	100	82	52	86.8%
GPT-4.1	100	100	85	81	37	80.7%
DeepSeek-V2 Chat	100	100	76	72	52	80.1%
Claude Sonnet 5	100	100	100	100	0	80.0%
MiniMax M2.5	100	100	78	78	44	80.0%
Z.AI GLM 5	95	89	87	81	45	79.4%
DeepSeek V3 (2025-03-24)	100	100	100	56	41	79.2%
Gemma 3 12B	100	100	100	45	44	77.8%
GPT-5.1	100	100	68	65	53	77.0%
Gemma 4 26B (Reasoning)	100	100	85	50	44	75.8%
Claude Opus 4.8 (Reasoning, Low)	100	100	62	60	54	75.0%
Qwen3 235B A22B Instruct 2507	100	100	98	77	0	74.9%
Claude Sonnet 5 (Reasoning)	100	100	93	81	0	74.8%
Claude Sonnet 5 (Reasoning, Low)	100	98	93	81	0	74.4%
Qwen3.7 Max	100	96	89	51	32	73.7%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	98	62	0	72.0%
Mistral Small 4	100	72	71	59	56	71.6%
MoonshotAI: Kimi K2.5	100	100	100	32	26	71.5%
GPT-4.1 Nano	100	100	100	56	0	71.1%
Xiaomi MIMO v2.5	100	87	76	61	29	70.5%
Ministral 3 14B	100	100	68	52	25	69.0%
GPT-4o Mini (temp=1)	100	100	87	50	0	67.3%
Gemma 3 4B	100	63	57	57	56	66.7%
Gemini 3.5 Flash (Reasoning)	100	85	66	44	37	66.4%
Claude Opus 4.6	100	93	65	37	34	65.8%
GPT-5	81	71	71	62	42	65.4%
Xiaomi MIMO v2.5 Pro	100	95	60	36	32	64.7%
Gemini 2.5 Flash	100	89	67	33	31	64.0%
Gemma 3 27B	100	89	84	45	0	63.7%
Qwen 3.5 Plus (2026-02-15)	100	100	78	41	0	63.6%
Claude Opus 4.7 (Reasoning)	100	97	65	53	0	63.0%
DeepSeek V4 Flash (Reasoning)	97	87	44	42	42	62.3%
MoonshotAI: Kimi K2.6	100	77	61	40	33	62.1%
Hermes 3 405B	100	100	67	42	0	61.7%
DeepSeek V3.2	100	100	68	37	0	61.1%
Claude Sonnet 4.6	100	84	78	40	0	60.6%
Mistral Large 3	100	93	59	51	0	60.6%
Ministral 8B	100	95	64	41	0	60.0%
Mistral Small 4 (Reasoning)	100	75	68	31	25	59.8%
Qwen 3.5 397B A17B	94	78	50	40	36	59.6%
Grok 4.5 (Reasoning, Low)	100	95	64	37	0	59.4%
GPT-5 Mini	90	75	74	57	0	59.2%
Gemini 2.5 Flash (Reasoning)	100	82	82	30	0	58.8%
o4 Mini	100	87	75	32	0	58.7%
Qwen 3.5 35B	96	86	74	37	0	58.7%
Gemini 3 Flash (Preview, Reasoning)	91	72	46	44	35	57.8%
Claude Sonnet 4.6 (Reasoning)	100	60	45	42	39	57.1%
Claude Sonnet 4	64	62	57	54	48	56.9%
DeepSeek V4 Pro (Reasoning)	100	100	60	23	0	56.4%
Mistral NeMO	100	75	71	35	0	56.3%
Claude Opus 4.8 (Reasoning)	100	65	63	51	0	55.8%
Z.AI GLM 5 Turbo	100	97	80	0	0	55.4%
MiniMax M3	100	90	45	41	0	55.3%
DeepSeek V4 Flash	74	72	48	48	34	55.1%
Ministral 3B	100	100	68	0	0	53.6%
Qwen 3 32B	100	55	43	40	31	53.6%
Aion 2.0	100	67	36	32	31	53.3%
Claude Opus 4.5	100	84	42	39	0	53.2%
GPT-5 Nano	100	64	49	33	19	53.0%
Claude Opus 4.6 (Reasoning)	100	69	59	35	0	52.8%
Gemini 2.5 Pro	100	100	35	29	0	52.8%
DeepSeek V4 Pro	89	78	51	39	0	51.4%
Qwen 3.6 Flash	100	77	29	25	24	50.9%
Claude Sonnet 4.5	100	100	53	0	0	50.6%
Claude Haiku 4.5	100	53	52	47	0	50.4%
Gemini 3.1 Flash Lite (Preview)	89	43	42	41	37	50.3%
Z.AI GLM 4.5	87	72	51	41	0	49.9%
Hermes 3 70B	74	62	56	55	0	49.4%
MiniMax M2.7	100	100	46	0	0	49.3%
DeepSeek V3 (2024-12-26)	100	56	43	41	0	47.9%
o4 Mini High	100	85	53	0	0	47.6%
Cydonia 24B V4.1	100	98	39	0	0	47.5%
Grok 4.20	89	65	40	22	18	46.9%
Gemini 3.1 Pro (Preview)	84	76	38	33	0	46.4%
Nemotron 3 Super	97	39	36	30	25	45.3%
Aion 3.0	97	70	57	0	0	44.8%
Z.AI GLM 4.6	100	72	46	0	0	43.6%
Gemini 3.1 Flash Lite (Reasoning)	84	45	41	39	0	41.8%
Z.AI GLM 4.5 Air	69	63	38	37	0	41.4%
GPT-4o, Aug. 6th (temp=1)	100	56	51	0	0	41.4%
Z.AI GLM 4.7	100	44	32	31	0	41.3%
Z.AI GLM 5.1	62	52	45	41	0	39.9%
GPT-4o Mini (temp=0)	53	50	48	46	0	39.5%
Qwen 3.6 35B	100	72	22	0	0	38.8%
Qwen3.6 Max Preview	68	51	26	21	19	37.1%
Gemma 4 26B	46	46	45	43	0	36.2%
Gemma 4 31B (Reasoning)	84	50	43	0	0	35.4%
ByteDance Seed 2.0 Mini	67	63	46	0	0	35.0%
Gemini 2.5 Flash Lite (Reasoning)	83	35	34	22	0	34.9%
Gemini 2.5 Flash Lite	100	37	35	0	0	34.5%
Ministral 3 3B	100	60	7	0	0	33.4%
Gemini 3 Flash (Preview)	85	78	0	0	0	32.8%
Qwen 3.6 27B	88	46	26	0	0	32.0%
ByteDance Seed 1.6	85	69	0	0	0	31.0%
Qwen 3.5 Flash	100	54	0	0	0	30.7%
Qwen 3.5 122B	44	43	39	27	0	30.6%
Z.AI GLM 4.7 Flash	64	42	41	0	0	29.3%
Qwen 3.5 9B	67	42	37	0	0	29.0%
Z.AI GLM 5.2 (Reasoning, High)	100	43	0	0	0	28.7%
Ministral 3 8B	87	45	7	0	0	27.7%
DeepSeek V3.1	54	46	37	0	0	27.3%
Grok 4.3 (Reasoning)	36	33	33	28	0	25.7%
Qwen 2.5 72B	74	46	0	0	0	23.9%
GPT-OSS 120B	46	42	31	0	0	23.7%
Claude Opus 4	44	36	35	0	0	23.0%
GPT-4o, Aug. 6th (temp=0)	61	49	0	0	0	21.9%
Cohere Command R+ (Aug. 2024)	62	46	0	0	0	21.5%
Mistral Small 3.2 24B	100	5	0	0	0	21.1%
Gemma 4 31B	100	0	0	0	0	20.0%
ByteDance Seed 2.0 Lite	100	0	0	0	0	20.0%
GPT-4.1 Mini	100	0	0	0	0	20.0%
Grok 4.3	37	34	23	0	0	18.8%
ByteDance Seed 1.6 Flash	32	31	25	0	0	17.6%
Aion 3.0 Mini	43	42	0	0	0	17.0%
Gemini 3.1 Flash Lite	42	39	0	0	0	16.3%
Nemotron 3 Nano	47	32	0	0	0	15.7%
Llama 3.1 70B	65	0	0	0	0	13.1%
WizardLM 2 8x22b	65	0	0	0	0	13.1%
Qwen 3.5 Plus (2026-04-20)	27	23	0	0	0	9.9%
Qwen 3.5 27B	36	0	0	0	0	7.2%
Inception Mercury 2	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Grok 4.5 (Reasoning, High)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.5 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
GPT-5.5 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
MiniMax M3	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemma 4 31B (Reasoning)	100	100	100	100	100	100.0%
GPT-5.5	100	100	100	100	100	100.0%
Claude Sonnet 5	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Grok 4.20	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Qwen3.6 Max Preview	100	100	100	100	98	99.6%
Mistral Large 3	100	100	100	100	95	99.0%
DeepSeek V3.2	100	100	100	100	95	98.9%
GPT-5.2	100	100	100	100	94	98.9%
Grok 4.5 (Reasoning, Low)	100	100	100	100	93	98.7%
GPT-5.4 Nano	100	100	100	100	92	98.5%
GPT-5.4 Nano (Reasoning)	100	100	100	100	88	97.7%
Aion 3.0 Mini	100	100	100	100	88	97.5%
GPT-5.1	100	100	100	100	87	97.3%
Gemini 3.5 Flash (Reasoning)	100	100	100	94	92	97.1%
Xiaomi MIMO v2.5 Pro	100	100	100	100	85	97.1%
Xiaomi MIMO v2.5	100	100	100	100	84	96.9%
Claude Opus 4.6 (Reasoning)	100	100	100	95	87	96.3%
Claude Sonnet 4	100	100	100	100	81	96.3%
Z.AI GLM 4.5	100	100	100	100	81	96.3%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	79	95.9%
Grok 4.20 (Reasoning)	100	100	100	100	78	95.6%
Claude Opus 4.7	100	100	98	91	85	95.0%
GPT-4o Mini (temp=1)	100	100	98	97	79	94.8%
DeepSeek V4 Flash (Reasoning)	100	100	100	100	73	94.7%
Ministral 3 8B	100	100	100	100	71	94.2%
Grok 4.3	100	100	100	85	82	93.6%
Mistral Small 4 (Reasoning)	100	100	100	98	69	93.5%
Qwen 3.5 Plus (2026-04-20)	100	100	100	99	63	92.3%
GPT-5 Mini	100	100	100	90	71	92.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	91	87	82	91.9%
Z.AI GLM 5.2 (Reasoning, High)	100	100	92	90	78	91.9%
DeepSeek V4 Pro (Reasoning)	100	100	100	99	58	91.5%
Qwen 3.6 Flash	100	100	100	86	71	91.4%
Claude Opus 4.6	100	100	100	100	55	90.9%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	92	56	89.6%
GPT-4o Mini (temp=0)	100	100	100	100	48	89.6%
Z.AI GLM 5	100	100	100	75	69	88.8%
Ministral 3 14B	100	100	100	79	64	88.7%
Mistral Small 4	100	100	95	79	67	88.1%
DeepSeek V3.1	100	100	100	69	67	87.1%
Claude Sonnet 4.5	100	100	100	76	60	87.1%
DeepSeek-V2 Chat	100	100	100	100	34	86.9%
Claude Opus 4.5	100	100	100	100	34	86.8%
Gemma 4 26B (Reasoning)	100	100	100	100	33	86.6%
ByteDance Seed 2.0 Mini	100	100	89	78	66	86.5%
Mistral NeMO	100	100	100	88	44	86.3%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	31	86.3%
GPT-4.1	100	100	100	100	31	86.1%
Z.AI GLM 4.7	100	97	87	85	60	85.8%
o4 Mini High	100	100	81	80	68	85.7%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	65	62	85.4%
Z.AI GLM 4.6	100	100	100	93	32	85.1%
Z.AI GLM 5.1	100	100	100	93	32	84.9%
Gemma 4 31B	100	100	82	77	61	84.0%
MoonshotAI: Kimi K2.6	100	100	100	70	50	83.9%
Ministral 3 3B	100	100	100	94	23	83.4%
Gemini 2.5 Flash Lite	100	100	100	74	39	82.6%
Gemini 3.1 Flash Lite	100	100	100	74	38	82.5%
Qwen 3.5 397B A17B	100	99	95	79	38	82.2%
GPT-5 Nano	100	99	88	81	41	81.7%
Claude Sonnet 4.6 (Reasoning)	100	100	90	60	58	81.6%
Mistral Medium 3.1	100	100	92	69	44	81.1%
GPT-4.1 Mini	100	100	89	72	42	80.7%
o4 Mini	100	100	80	70	52	80.3%
DeepSeek V3 (2025-03-24)	100	100	100	100	0	80.0%
Qwen 3 32B	100	100	100	100	0	80.0%
Hermes 3 70B	100	100	100	100	0	80.0%
ByteDance Seed 1.6 Flash	100	100	71	65	63	79.9%
Qwen 3.6 35B	100	100	99	57	43	79.8%
DeepSeek V3 (2024-12-26)	100	100	100	93	0	78.5%
Claude Opus 4.7 (Reasoning)	100	100	100	48	40	77.7%
Aion 2.0	100	100	70	67	47	76.8%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	100	54	29	76.5%
Gemini 2.5 Pro	100	100	100	56	24	76.0%
Qwen 3.5 Flash	100	82	79	65	53	75.8%
Nemotron 3 Super	100	95	76	62	40	74.6%
Gemini 3 Flash (Preview)	87	86	84	58	56	74.2%
Gemini 3.1 Flash Lite (Reasoning)	83	78	72	70	66	73.7%
Claude Opus 4	100	98	88	74	0	72.0%
Z.AI GLM 4.7 Flash	100	100	85	74	0	71.7%
DeepSeek V4 Flash	100	100	92	34	31	71.4%
Qwen 3.6 27B	100	100	80	52	22	70.7%
Cydonia 24B V4.1	100	100	82	69	0	70.2%
Aion 3.0	100	100	68	63	0	66.2%
Gemini 3.1 Pro (Preview)	100	85	78	43	23	65.9%
Claude Opus 4.8 (Reasoning)	98	93	88	51	0	65.8%
Gemini 3.1 Flash Lite (Preview)	100	85	65	41	36	65.5%
Ministral 8B	100	100	78	47	0	64.9%
Arcee AI: Trinity Mini	100	100	91	31	0	64.6%
ByteDance Seed 2.0 Lite	100	94	64	62	0	63.9%
Qwen3.7 Max	100	79	60	47	32	63.6%
DeepSeek V4 Pro	84	71	67	64	30	63.3%
MiniMax M2.7	100	100	50	27	26	60.8%
GPT-5	98	60	52	49	43	60.5%
Claude Sonnet 5 (Reasoning)	100	74	67	61	0	60.3%
Cohere Command R+ (Aug. 2024)	100	74	52	41	32	59.8%
Qwen 3.5 35B	99	76	68	28	27	59.5%
Hermes 3 405B	100	79	60	57	0	59.3%
Claude Sonnet 4.6	100	84	63	43	0	58.0%
Claude Opus 4.8 (Reasoning, Low)	100	98	48	42	0	57.5%
Qwen 3.5 122B	100	99	49	26	0	54.7%
Qwen 3.5 9B	99	78	63	22	0	52.2%
Z.AI GLM 4.5 Air	84	58	51	37	29	51.7%
Qwen 3.5 27B	100	52	51	27	26	51.1%
MoonshotAI: Kimi K2.5	100	100	50	0	0	49.9%
Grok 4.3 (Reasoning)	100	63	43	33	0	47.8%
Gemma 4 26B	100	96	31	0	0	45.3%
Ministral 3B	100	68	48	0	0	43.2%
Llama 3.1 70B	74	67	62	0	0	40.5%
GPT-4o, Aug. 6th (temp=0)	100	94	0	0	0	38.8%
Nemotron 3 Nano	100	61	31	0	0	38.3%
ByteDance Seed 1.6	100	91	0	0	0	38.3%
Mistral Small 3.2 24B	100	43	28	5	0	35.1%
GPT-OSS 120B	62	52	38	22	0	34.6%
WizardLM 2 8x22b	58	56	0	0	0	23.0%
Qwen 2.5 72B	88	0	0	0	0	17.5%
Inception Mercury 2	36	25	23	0	0	16.6%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.5 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Small 4	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	95	99.0%
Grok 4.20	100	100	100	100	89	97.9%
GPT-5.4 Nano (Reasoning)	100	100	100	100	83	96.6%
Qwen3 235B A22B Instruct 2507	100	100	100	100	82	96.5%
Ministral 3B	100	100	100	98	81	95.9%
GPT-5.5 (Reasoning, Low)	100	100	100	93	84	95.4%
GPT-5.4	100	100	100	100	76	95.2%
DeepSeek V4 Flash (Reasoning)	100	100	100	100	69	93.9%
GPT-5.4 (Reasoning)	100	100	100	96	72	93.6%
Mistral Medium 3.1	100	100	100	100	67	93.3%
Mistral Small 4 (Reasoning)	100	100	100	100	63	92.6%
Gemini 2.5 Pro	100	100	100	100	58	91.7%
Claude Opus 4.7	100	100	100	79	76	91.0%
GPT-5.4 (Reasoning, Low)	100	100	100	98	51	89.7%
Writer: Palmyra X5	100	100	100	93	56	89.6%
GPT-5.5	100	100	97	89	62	89.6%
GPT-5.1	100	100	94	93	60	89.3%
Gemma 3 4B	100	100	100	93	51	88.8%
Ministral 3 14B	100	100	100	87	56	88.6%
Grok 4.5 (Reasoning, High)	100	100	100	100	34	86.8%
Claude Haiku 4.5	100	98	95	91	46	86.1%
GPT-5.2	100	93	84	79	68	84.9%
MiniMax M3	100	100	100	76	47	84.5%
DeepSeek V4 Flash	100	100	100	77	37	82.7%
Z.AI GLM 5.1	100	100	87	81	43	82.2%
o4 Mini	100	100	100	68	33	80.3%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	93	0	78.5%
Claude Opus 4.7 (Reasoning)	100	100	69	63	58	78.2%
Aion 3.0 Mini	100	100	100	48	38	77.2%
Grok 4.20 (Reasoning)	100	100	100	54	31	76.9%
GPT-5 Nano	100	100	95	67	20	76.5%
GPT-5.4 Nano (Reasoning, Low)	100	97	82	71	29	75.7%
MiniMax M2.5	100	100	93	40	39	74.6%
MiniMax M2.7	100	100	100	41	31	74.3%
GPT-4.1	100	82	76	75	37	74.0%
Ministral 8B	100	100	100	34	32	73.2%
Z.AI GLM 5.2 (Reasoning, High)	100	96	66	59	42	72.6%
Claude Opus 4.6	100	100	70	60	32	72.5%
Llama 3.1 70B	100	100	100	60	0	71.8%
Claude Opus 4	100	88	78	48	44	71.7%
DeepSeek V3 (2025-03-24)	100	100	93	62	0	70.9%
Claude Sonnet 4	91	88	87	48	38	70.4%
Claude Sonnet 4.6 (Reasoning)	100	91	84	75	0	70.1%
o4 Mini High	100	78	67	66	34	69.2%
Qwen3.6 Max Preview	100	95	61	50	37	68.5%
Claude Opus 4.8 (Reasoning)	100	100	74	65	0	67.9%
MoonshotAI: Kimi K2.6	100	100	64	39	29	66.4%
Grok 4.5 (Reasoning, Low)	100	100	92	35	0	65.4%
DeepSeek V4 Pro (Reasoning)	100	84	56	54	32	65.0%
Mistral Large 3	100	100	79	46	0	65.0%
Claude Sonnet 4.5	100	89	84	51	0	64.9%
GPT-5.4 Nano	100	88	81	29	23	64.3%
Qwen 3 32B	100	65	58	56	41	64.1%
DeepSeek-V2 Chat	100	80	55	52	33	64.1%
Ministral 3 8B	100	99	96	20	0	63.1%
Qwen3.7 Max	100	83	56	51	25	62.8%
ByteDance Seed 1.6 Flash	100	100	37	32	32	60.1%
Gemini 2.5 Flash Lite	100	96	42	34	28	60.1%
DeepSeek V3 (2024-12-26)	100	100	56	40	0	59.3%
Claude Opus 4.6 (Reasoning)	100	78	58	57	0	58.9%
Claude Sonnet 5 (Reasoning, Low)	100	100	93	0	0	58.5%
Z.AI GLM 5	100	90	60	34	0	56.7%
Claude Opus 4.5	88	88	67	40	0	56.4%
Gemma 3 12B	100	78	67	37	0	56.2%
GPT-5	100	71	45	44	18	55.6%
Z.AI GLM 5 Turbo	95	93	85	0	0	54.8%
Mistral NeMO	100	100	72	0	0	54.5%
Claude Sonnet 4.6	100	81	52	37	0	54.1%
Z.AI GLM 4.7 Flash	98	69	35	33	32	53.2%
Gemini 2.5 Flash	100	78	56	31	0	53.1%
Qwen 3.6 35B	100	68	64	27	0	51.8%
Ministral 3 3B	100	61	56	41	0	51.7%
Gemini 3.5 Flash (Reasoning)	100	85	37	34	0	51.2%
Nemotron 3 Super	70	52	47	44	42	51.2%
Gemini 3.1 Pro (Preview)	100	72	53	29	0	50.9%
Nemotron 3 Nano	100	90	58	0	0	49.5%
Claude Sonnet 5	100	74	72	0	0	49.3%
Aion 2.0	100	64	54	25	0	48.6%
Qwen 3.5 Plus (2026-02-15)	100	51	47	44	0	48.4%
Xiaomi MIMO v2.5	84	81	41	34	0	48.1%
Hermes 3 405B	100	76	62	0	0	47.5%
DeepSeek V3.2	100	98	36	0	0	46.8%
MoonshotAI: Kimi K2.5	100	81	48	0	0	45.8%
Qwen 3.6 Flash	90	54	41	31	0	43.1%
Gemini 2.5 Flash Lite (Reasoning)	70	61	53	31	0	42.9%
GPT-4.1 Nano	100	60	55	0	0	42.8%
GPT-4o, Aug. 6th (temp=1)	100	57	56	0	0	42.8%
DeepSeek V4 Pro	91	38	36	34	0	39.7%
GPT-4o Mini (temp=1)	52	50	49	45	0	39.2%
Qwen 3.5 Plus (2026-04-20)	100	54	40	0	0	38.7%
Grok 4.3 (Reasoning)	78	63	53	0	0	38.7%
Z.AI GLM 4.6	85	42	37	28	0	38.5%
Gemini 3.1 Flash Lite	81	77	33	0	0	38.1%
Qwen 3.5 397B A17B	68	52	36	33	0	37.8%
Xiaomi MIMO v2.5 Pro	100	29	28	28	0	37.1%
DeepSeek V3.1	99	56	29	0	0	36.8%
Gemini 2.5 Flash (Reasoning)	100	49	29	0	0	35.7%
Mistral Small 3.2 24B	100	78	0	0	0	35.6%
Gemma 3 27B	71	66	41	0	0	35.6%
Qwen 3.5 27B	69	50	31	28	0	35.5%
Gemini 3.1 Flash Lite (Reasoning)	65	40	37	32	0	34.7%
Gemma 4 26B (Reasoning)	87	87	0	0	0	34.6%
Gemini 3 Flash (Preview)	91	40	39	0	0	34.0%
GPT-5 Mini	100	23	22	22	0	33.4%
Grok 4.3	88	48	30	0	0	33.1%
Qwen 3.6 27B	100	33	32	0	0	32.9%
Qwen 3.5 35B	100	62	0	0	0	32.3%
GPT-4o Mini (temp=0)	60	51	48	0	0	31.5%
Z.AI GLM 4.5 Air	82	41	31	0	0	30.9%
Cydonia 24B V4.1	72	42	40	0	0	30.6%
Gemma 4 31B	51	51	45	0	0	29.4%
ByteDance Seed 2.0 Mini	100	42	0	0	0	28.4%
Gemma 4 26B	50	49	43	0	0	28.3%
Aion 3.0	77	65	0	0	0	28.3%
GPT-4.1 Mini	67	61	0	0	0	25.5%
Z.AI GLM 4.5	78	46	0	0	0	24.6%
WizardLM 2 8x22b	72	48	0	0	0	24.2%
Qwen 3.5 Flash	54	36	29	0	0	23.8%
Gemini 3.5 Flash (Reasoning, Minimal)	46	37	35	0	0	23.7%
Qwen 2.5 72B	64	51	0	0	0	22.9%
Z.AI GLM 4.7	35	35	31	0	0	20.3%
Hermes 3 70B	100	0	0	0	0	20.0%
Qwen 3.5 122B	45	41	0	0	0	17.1%
GPT-4o, Aug. 6th (temp=0)	44	40	0	0	0	16.9%
Gemini 3 Flash (Preview, Reasoning)	82	0	0	0	0	16.5%
GPT-OSS 120B	44	37	0	0	0	16.2%
Qwen 3.5 9B	39	36	0	0	0	15.1%
Gemini 3.1 Flash Lite (Preview)	37	36	0	0	0	14.7%
Inception Mercury 2	39	32	0	0	0	14.1%
Cohere Command R+ (Aug. 2024)	64	0	0	0	0	12.8%
Arcee AI: Trinity Mini	52	0	0	0	0	10.4%
Gemma 4 31B (Reasoning)	49	0	0	0	0	9.8%
Claude Sonnet 5 (Reasoning)	0	0	0	0	0	0.0%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
ByteDance Seed 2.0 Lite	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Writer: Palmyra X5	100	100	100	100	77	95.3%
Ministral 3 14B	100	100	100	100	67	93.3%
GPT-5.5 (Reasoning)	100	100	97	92	65	90.9%
Mistral Medium 3.1	100	100	92	88	67	89.5%
GPT-5.4	100	100	100	78	59	87.5%
Grok 4.20	100	90	88	81	76	86.9%
Claude Sonnet 4.5	100	100	100	93	38	86.1%
Mistral Large 3	100	100	100	81	44	85.1%
GPT-5.4 (Reasoning, Low)	100	100	84	73	56	82.6%
Qwen 3.6 27B	100	100	100	69	35	80.8%
Qwen 3 32B	100	93	86	63	62	80.7%
GPT-5.4 Mini	100	100	90	60	53	80.6%
Z.AI GLM 5 Turbo	100	100	89	80	33	80.5%
Qwen 3.5 397B A17B	100	100	77	63	62	80.4%
Grok 4.5 (Reasoning, High)	100	100	100	58	40	79.6%
GPT-5.4 (Reasoning)	100	86	78	72	59	79.0%
Claude Sonnet 4.6	100	100	97	53	42	78.2%
GPT-4.1 Nano	100	91	69	64	57	76.5%
Claude Opus 4	100	100	100	82	0	76.5%
Mistral Large 2	100	100	95	85	0	76.1%
GPT-5.1	100	88	67	63	57	75.1%
Claude Sonnet 5 (Reasoning, Low)	100	100	88	88	0	75.1%
Gemini 3 Flash (Preview)	100	100	100	75	0	75.0%
Qwen 3.6 Flash	100	78	71	68	57	74.7%
Mistral Small 4 (Reasoning)	99	93	72	72	38	74.7%
Gemma 3 12B	100	100	93	41	38	74.2%
Mistral Small 4	96	81	78	72	41	73.6%
GPT-5.4 Mini (Reasoning, Low)	100	89	85	51	40	73.1%
DeepSeek-V2 Chat	100	93	68	62	41	72.6%
Claude Opus 4.6 (Reasoning)	100	100	90	72	0	72.4%
Gemma 4 31B	100	89	85	45	42	72.3%
Qwen 3.6 35B	100	100	81	44	34	71.9%
GPT-5.4 Mini (Reasoning)	100	100	87	48	25	71.9%
GPT-5.5	96	76	75	59	52	71.5%
DeepSeek V3 (2025-03-24)	100	100	81	72	0	70.8%
MoonshotAI: Kimi K2.6	100	100	92	62	0	70.7%
GPT-5.5 (Reasoning, Low)	100	84	83	46	39	70.3%
Gemini 2.5 Flash Lite (Reasoning)	100	69	64	61	51	68.9%
Z.AI GLM 5.1	100	100	63	40	36	67.9%
Ministral 8B	100	100	90	46	0	67.2%
GPT-5.4 Nano (Reasoning)	100	95	78	42	17	66.6%
Gemini 2.5 Flash (Reasoning)	100	100	59	39	32	65.9%
Gemma 4 26B (Reasoning)	100	95	46	45	41	65.4%
ByteDance Seed 1.6 Flash	100	90	65	49	21	65.0%
MiniMax M2.5	100	97	83	41	0	64.1%
Gemma 4 31B (Reasoning)	100	84	48	45	41	63.6%
Gemini 3.1 Pro (Preview)	100	94	85	34	0	62.7%
Ministral 3 3B	100	84	81	37	4	61.2%
DeepSeek V4 Pro (Reasoning)	98	88	85	35	0	61.2%
GPT-5.4 Nano (Reasoning, Low)	88	69	58	45	44	60.7%
GPT-5.4 Nano	100	68	64	55	16	60.4%
GPT-4o, Aug. 6th (temp=1)	100	100	100	0	0	60.0%
Gemma 4 26B	93	82	81	41	0	59.4%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	40	39	0	55.9%
MiniMax M2.7	100	50	44	42	42	55.6%
Qwen3 235B A22B Instruct 2507	100	52	44	42	33	54.1%
Ministral 3B	100	62	60	46	0	53.5%
Claude Sonnet 4.6 (Reasoning)	99	89	43	37	0	53.5%
Qwen 3.5 Plus (2026-02-15)	100	85	42	39	0	53.3%
Claude Opus 4.7	100	62	56	48	0	53.2%
Z.AI GLM 4.7 Flash	69	67	60	37	29	52.7%
GPT-5	100	85	39	38	0	52.5%
Qwen3.6 Max Preview	90	60	56	36	21	52.5%
Gemma 3 4B	58	52	51	49	48	51.8%
Gemini 3.1 Flash Lite (Preview)	100	90	35	33	0	51.8%
Grok 4.3	100	62	60	35	0	51.3%
Qwen 3.5 27B	74	70	66	45	0	51.0%
Z.AI GLM 4.7	95	71	56	31	0	50.5%
GPT-5.2	100	67	45	17	16	49.1%
Claude Sonnet 5	100	76	68	0	0	48.8%
Mistral NeMO	100	76	63	0	0	47.7%
Z.AI GLM 5.2 (Reasoning, High)	87	61	50	41	0	47.6%
o4 Mini	100	100	36	0	0	47.2%
Grok 4.5 (Reasoning, Low)	80	57	54	43	0	46.9%
Qwen 3.5 122B	73	46	45	38	32	46.7%
Claude Opus 4.6	84	56	32	31	29	46.6%
Cohere Command R+ (Aug. 2024)	93	85	51	0	0	45.9%
Grok 4.20 (Reasoning)	85	58	55	28	0	45.5%
Gemini 2.5 Pro	100	33	31	31	30	45.0%
Qwen3.7 Max	100	90	31	0	0	44.1%
MiniMax M3	78	51	43	42	0	42.7%
Qwen 3.5 9B	83	48	44	39	0	42.6%
Claude Opus 4.5	100	73	36	0	0	41.9%
GPT-4.1 Mini	100	56	51	0	0	41.4%
Claude Opus 4.7 (Reasoning)	100	60	47	0	0	41.3%
Claude Opus 4.8 (Reasoning, Low)	100	60	45	0	0	40.9%
Gemini 3 Flash (Preview, Reasoning)	100	63	40	0	0	40.5%
Z.AI GLM 5	100	69	31	0	0	40.0%
DeepSeek V4 Pro	67	63	36	32	0	39.6%
o4 Mini High	100	67	29	0	0	39.2%
Ministral 3 8B	100	57	36	1	0	38.9%
Claude Opus 4.8 (Reasoning)	100	94	0	0	0	38.8%
Qwen 3.5 35B	100	64	27	0	0	38.3%
Hermes 3 405B	100	90	0	0	0	38.0%
MoonshotAI: Kimi K2.5	90	58	40	0	0	37.7%
Claude Sonnet 5 (Reasoning)	100	88	0	0	0	37.5%
Aion 3.0	62	58	35	32	0	37.4%
Qwen 3.5 Flash	68	41	38	32	0	36.0%
Aion 3.0 Mini	60	52	34	32	0	35.9%
Gemini 3.1 Flash Lite	100	78	0	0	0	35.5%
Gemini 3.1 Flash Lite (Reasoning)	100	38	37	0	0	35.1%
Gemma 3 27B	100	39	36	0	0	35.0%
Nemotron 3 Nano	100	73	0	0	0	34.7%
DeepSeek V4 Flash (Reasoning)	100	41	32	0	0	34.5%
GPT-4o Mini (temp=1)	57	56	55	0	0	33.5%
Xiaomi MIMO v2.5 Pro	71	37	29	28	0	32.9%
DeepSeek V4 Flash	98	66	0	0	0	32.8%
DeepSeek V3.1	77	44	39	0	0	31.9%
Gemini 3.5 Flash (Reasoning)	43	43	38	33	0	31.3%
GPT-4.1	78	40	34	0	0	30.4%
DeepSeek V3.2	60	58	33	0	0	30.4%
Gemini 2.5 Flash	44	33	31	30	0	27.6%
Nemotron 3 Super	51	47	27	0	0	25.0%
Claude Sonnet 4	62	61	0	0	0	24.5%
GPT-5 Nano	57	49	15	0	0	24.2%
Qwen 3.5 Plus (2026-04-20)	62	29	28	0	0	23.8%
DeepSeek V3 (2024-12-26)	83	33	0	0	0	23.3%
Grok 4.3 (Reasoning)	61	53	0	0	0	22.7%
GPT-4o, Aug. 6th (temp=0)	60	54	0	0	0	22.7%
Z.AI GLM 4.6	41	37	35	0	0	22.5%
Claude Haiku 4.5	56	51	0	0	0	21.4%
Cydonia 24B V4.1	61	46	0	0	0	21.3%
Xiaomi MIMO v2.5	62	41	0	0	0	20.6%
Arcee AI: Trinity Mini	98	0	0	0	0	19.6%
Z.AI GLM 4.5 Air	95	0	0	0	0	19.0%
Gemini 2.5 Flash Lite	42	36	0	0	0	15.6%
ByteDance Seed 2.0 Lite	65	0	0	0	0	13.1%
Llama 3.1 70B	56	0	0	0	0	11.3%
ByteDance Seed 1.6	50	0	0	0	0	10.0%
ByteDance Seed 2.0 Mini	48	0	0	0	0	9.5%
Z.AI GLM 4.5	43	0	0	0	0	8.5%
Aion 2.0	37	0	0	0	0	7.3%
GPT-OSS 120B	26	0	0	0	0	5.2%
GPT-5 Mini	21	0	0	0	0	4.3%
Mistral Small 3.2 24B	3	0	0	0	0	0.5%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%
Hermes 3 70B	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.5 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Grok 4.20	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	98	99.6%
GPT-5.4	100	100	100	100	80	96.0%
GPT-5.5	100	100	100	100	78	95.7%
Aion 3.0	100	100	100	88	74	92.5%
Grok 4.5 (Reasoning, Low)	100	100	100	87	67	90.8%
GPT-4.1	100	100	100	89	65	90.7%
GPT-5.5 (Reasoning)	100	100	100	100	49	89.8%
Qwen3 235B A22B Instruct 2507	100	100	100	100	49	89.8%
Grok 4.5 (Reasoning, High)	100	100	100	91	58	89.8%
Mistral Large 3	100	100	100	100	43	88.5%
GPT-5.1	100	100	100	79	62	88.3%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	40	87.9%
GPT-5.4 (Reasoning)	100	100	90	88	56	86.9%
Mistral Medium 3.1	100	100	83	74	62	83.7%
GPT-5.4 Nano (Reasoning)	100	100	85	68	65	83.5%
Z.AI GLM 5.1	100	100	100	80	37	83.4%
MoonshotAI: Kimi K2.6	100	98	91	67	56	82.5%
Gemini 3 Flash (Preview)	100	100	90	83	38	82.3%
Qwen 3.6 35B	100	100	100	83	27	82.1%
Mistral Small 4	100	89	83	78	60	81.9%
GPT-5.4 Nano	98	90	90	63	60	80.2%
Z.AI GLM 5 Turbo	100	100	100	100	0	80.0%
MiniMax M3	100	100	100	100	0	80.0%
Writer: Palmyra X5	100	100	100	100	0	80.0%
Claude Opus 4.7	100	100	72	69	57	79.9%
Claude Opus 4	100	88	73	72	64	79.4%
Claude Sonnet 5 (Reasoning, Low)	100	100	98	98	0	79.2%
Claude Opus 4.8 (Reasoning)	100	100	81	57	56	78.9%
DeepSeek V4 Pro	100	100	100	63	26	77.9%
Claude Opus 4.6	100	100	98	79	0	75.5%
Qwen 3.6 Flash	100	94	83	66	34	75.3%
MiniMax M2.5	100	100	94	46	33	74.5%
GPT-5 Mini	100	100	89	47	37	74.5%
Ministral 3 14B	100	100	71	57	44	74.5%
Gemini 3.1 Pro (Preview)	100	100	100	69	0	73.8%
Mistral Large 2	100	100	100	44	25	73.8%
DeepSeek V3 (2024-12-26)	100	84	81	52	49	73.4%
Qwen3.7 Max	100	94	91	46	29	72.2%
Grok 4.20 (Reasoning)	100	100	100	31	29	71.9%
Ministral 3 3B	100	100	61	53	46	71.8%
GPT-5.2	100	82	70	54	51	71.4%
Grok 4.3	100	100	100	51	0	70.3%
o4 Mini	100	100	81	66	0	69.5%
Claude Sonnet 4.6 (Reasoning)	100	100	91	56	0	69.4%
Claude Opus 4.6 (Reasoning)	100	100	80	65	0	69.1%
Aion 2.0	100	78	72	56	40	69.0%
Claude Sonnet 4.5	100	100	100	41	0	68.1%
Gemma 4 26B	100	98	90	51	0	67.9%
Z.AI GLM 4.6	100	78	70	55	35	67.7%
Ministral 3 8B	96	96	77	61	6	67.1%
o4 Mini High	100	100	69	65	0	67.0%
Xiaomi MIMO v2.5	100	100	67	35	32	66.8%
DeepSeek V4 Flash	100	73	72	43	43	66.3%
Gemini 2.5 Flash Lite (Reasoning)	88	83	73	38	33	63.2%
GPT-5 Nano	100	79	57	55	22	62.7%
MiniMax M2.7	100	100	76	36	0	62.3%
Gemma 4 26B (Reasoning)	100	100	56	55	0	62.2%
Z.AI GLM 5	100	100	73	37	0	62.1%
Gemma 3 27B	78	75	68	45	42	61.7%
DeepSeek V3 (2025-03-24)	100	90	63	56	0	61.7%
Gemini 2.5 Pro	100	82	62	58	0	60.4%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	0	0	60.0%
Qwen 3.5 Plus (2026-04-20)	100	100	96	0	0	59.2%
Gemini 3.5 Flash (Reasoning, Minimal)	100	98	49	48	0	58.9%
Z.AI GLM 5.2 (Reasoning, High)	100	100	55	31	0	57.2%
Claude Opus 4.5	100	100	44	42	0	57.2%
Cohere Command R+ (Aug. 2024)	100	100	51	34	0	56.9%
DeepSeek V4 Pro (Reasoning)	100	100	53	31	0	56.8%
GPT-5	100	55	50	41	36	56.4%
Gemini 3.5 Flash (Reasoning)	100	100	42	37	0	55.8%
Qwen 3.5 397B A17B	100	64	63	51	0	55.7%
Qwen 3.5 Flash	100	71	67	37	0	54.9%
Claude Sonnet 5 (Reasoning)	100	100	67	0	0	53.3%
Ministral 3B	100	100	65	0	0	53.1%
Z.AI GLM 4.7	64	63	57	52	27	52.9%
Qwen3.6 Max Preview	100	100	42	22	0	52.8%
GPT-4.1 Mini	100	100	63	0	0	52.6%
GPT-4.1 Nano	100	100	63	0	0	52.6%
Gemini 3.1 Flash Lite (Reasoning)	80	71	70	39	0	52.1%
DeepSeek V3.2	77	76	76	32	0	52.0%
Claude Haiku 4.5	100	55	51	51	0	51.4%
Claude Opus 4.7 (Reasoning)	100	97	60	0	0	51.2%
Qwen 3.5 35B	99	53	42	29	23	49.2%
DeepSeek V3.1	100	72	38	35	0	49.0%
Qwen 3.6 27B	100	69	43	33	0	48.9%
Gemma 3 4B	100	95	48	0	0	48.6%
Claude Sonnet 4	100	100	40	0	0	47.9%
Gemini 2.5 Flash (Reasoning)	100	52	32	29	25	47.8%
Qwen 3.5 Plus (2026-02-15)	100	93	46	0	0	47.7%
DeepSeek V4 Flash (Reasoning)	100	48	46	43	0	47.3%
Qwen 3.5 122B	80	73	46	34	0	46.7%
Nemotron 3 Super	100	67	57	0	0	44.9%
ByteDance Seed 1.6 Flash	100	80	36	0	0	43.3%
Xiaomi MIMO v2.5 Pro	92	60	35	28	0	42.9%
Aion 3.0 Mini	100	76	37	0	0	42.6%
Z.AI GLM 4.7 Flash	78	72	30	26	0	41.2%
Gemini 3 Flash (Preview, Reasoning)	87	78	35	0	0	40.1%
Claude Sonnet 5	100	100	0	0	0	40.0%
Gemma 4 31B (Reasoning)	61	46	46	45	0	39.4%
GPT-4o Mini (temp=1)	100	51	44	0	0	39.0%
Arcee AI: Trinity Mini	100	90	0	0	0	38.0%
Qwen 3 32B	100	56	33	0	0	37.7%
Gemini 3.1 Flash Lite	100	78	0	0	0	35.5%
DeepSeek-V2 Chat	93	44	39	0	0	35.2%
Grok 4.3 (Reasoning)	100	76	0	0	0	35.2%
Llama 3.1 70B	100	72	0	0	0	34.5%
Gemini 2.5 Flash	84	56	26	0	0	33.4%
GPT-4o, Aug. 6th (temp=0)	100	65	0	0	0	33.1%
GPT-4o, Aug. 6th (temp=1)	100	65	0	0	0	33.1%
Gemini 2.5 Flash Lite	96	40	28	0	0	32.8%
Cydonia 24B V4.1	56	53	51	0	0	31.8%
Qwen 3.5 27B	95	39	22	0	0	31.2%
Hermes 3 70B	74	71	0	0	0	29.0%
Qwen 2.5 72B	53	48	44	0	0	29.0%
Ministral 8B	100	41	0	0	0	28.2%
MoonshotAI: Kimi K2.5	100	36	0	0	0	27.2%
Hermes 3 405B	78	51	0	0	0	25.8%
Z.AI GLM 4.5	88	40	0	0	0	25.6%
ByteDance Seed 2.0 Mini	69	56	0	0	0	25.0%
WizardLM 2 8x22b	69	54	0	0	0	24.6%
Z.AI GLM 4.5 Air	71	48	0	0	0	23.8%
Gemini 3.1 Flash Lite (Preview)	70	43	0	0	0	22.7%
Gemma 4 31B	57	53	0	0	0	22.1%
Claude Sonnet 4.6	100	0	0	0	0	20.0%
Mistral NeMO	100	0	0	0	0	20.0%
GPT-4o Mini (temp=0)	93	0	0	0	0	18.5%
Gemma 3 12B	93	0	0	0	0	18.5%
GPT-OSS 120B	58	26	0	0	0	17.0%
ByteDance Seed 2.0 Lite	76	0	0	0	0	15.2%
ByteDance Seed 1.6	65	0	0	0	0	13.1%
Qwen 3.5 9B	56	0	0	0	0	11.3%
Mistral Small 3.2 24B	45	0	0	0	0	9.0%
Nemotron 3 Nano	40	0	0	0	0	7.9%
Inception Mercury 2	32	0	0	0	0	6.3%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.5 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Opus 4.7	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Claude Opus 4.8 (Reasoning)	100	100	100	100	95	99.0%
GPT-5.5 (Reasoning, Low)	100	100	100	100	94	98.8%
GPT-5.4 Mini	100	100	100	100	90	98.0%
GPT-5.4	100	100	100	100	89	97.7%
Mistral Medium 3.1	100	100	100	100	85	97.1%
Claude Sonnet 5 (Reasoning)	100	100	100	100	81	96.3%
Grok 4.5 (Reasoning, High)	100	100	100	100	77	95.3%
GPT-5.4 (Reasoning)	100	100	100	100	76	95.2%
Claude Opus 4.7 (Reasoning)	100	100	100	89	85	94.9%
Mistral Small 4 (Reasoning)	100	100	100	93	78	94.3%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	100	67	93.3%
GPT-5.2	100	100	100	89	77	93.2%
GPT-5.5	100	100	100	83	82	93.1%
Gemini 2.5 Flash Lite	100	100	100	88	71	91.7%
Claude Haiku 4.5	100	100	100	94	64	91.6%
GPT-4.1 Mini	100	100	100	100	56	91.1%
MoonshotAI: Kimi K2.6	100	100	100	84	69	90.8%
GPT-5.4 Mini (Reasoning)	100	100	100	100	53	90.7%
GPT-4.1 Nano	100	100	100	100	49	89.8%
Gemma 3 4B	100	100	100	90	58	89.7%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	43	88.6%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	79	63	88.5%
Claude Opus 4.8 (Reasoning, Low)	100	100	98	91	52	88.2%
Mistral Small 4	100	100	100	100	38	87.7%
GPT-5.4 Nano	100	96	89	81	73	87.7%
GPT-5.4 Nano (Reasoning)	100	100	100	90	48	87.5%
Mistral Large 3	100	100	100	79	57	87.4%
Mistral NeMO	100	100	100	93	39	86.3%
Claude Sonnet 5	100	100	100	71	57	85.7%
Ministral 3 8B	100	100	94	78	39	82.1%
Claude Sonnet 4.6 (Reasoning)	100	100	100	69	39	81.6%
Aion 3.0 Mini	100	100	93	68	46	81.6%
Grok 4.20 (Reasoning)	100	100	100	83	25	81.5%
Z.AI GLM 5.1	100	100	92	78	34	80.9%
DeepSeek-V2 Chat	100	98	78	71	52	79.7%
MiniMax M2.7	100	100	100	96	0	79.2%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	89	74	33	79.1%
GPT-5.1	100	100	100	68	27	79.1%
Gemini 2.5 Flash (Reasoning)	100	100	73	65	52	78.2%
Claude Sonnet 4.5	100	100	100	89	0	77.8%
Gemma 3 27B	100	100	92	63	31	77.2%
MiniMax M3	100	100	76	75	34	77.0%
MiniMax M2.5	100	100	100	41	39	76.0%
Grok 4.5 (Reasoning, Low)	100	97	90	88	0	75.1%
Grok 4.20	99	89	81	70	25	73.0%
Aion 3.0	100	100	77	56	30	72.6%
ByteDance Seed 1.6	100	100	100	62	0	72.3%
Z.AI GLM 5.2 (Reasoning, High)	100	94	72	58	31	71.1%
Gemini 3.5 Flash (Reasoning)	100	100	81	39	32	70.5%
DeepSeek V3 (2025-03-24)	100	100	100	52	0	70.4%
Ministral 3 3B	100	100	90	37	23	70.1%
GPT-4.1	100	100	91	60	0	70.1%
Nemotron 3 Super	99	85	82	46	36	69.8%
Qwen 3.6 27B	100	100	94	28	21	68.5%
Gemma 4 26B (Reasoning)	100	100	98	41	0	67.8%
Claude Sonnet 4.6	100	100	94	44	0	67.8%
Grok 4.3	100	100	100	39	0	67.8%
Ministral 8B	100	100	69	34	31	66.7%
Qwen 3.5 397B A17B	89	78	63	54	45	66.0%
Ministral 3B	95	79	65	56	33	65.7%
Z.AI GLM 5 Turbo	100	94	65	37	32	65.5%
Ministral 3 14B	100	100	52	42	32	65.1%
Xiaomi MIMO v2.5 Pro	100	100	100	25	0	65.1%
DeepSeek V4 Flash	100	100	62	41	0	60.7%
MoonshotAI: Kimi K2.5	100	100	61	37	0	59.4%
o4 Mini High	100	81	68	24	23	59.3%
DeepSeek V4 Flash (Reasoning)	100	72	63	60	0	59.1%
DeepSeek V3.2	86	78	56	50	25	59.0%
Claude Opus 4	100	87	72	33	0	58.4%
Z.AI GLM 5	100	90	62	38	0	58.0%
Gemini 2.5 Flash	100	100	49	38	0	57.5%
Z.AI GLM 4.7 Flash	100	69	54	35	28	57.2%
Gemini 2.5 Flash Lite (Reasoning)	100	88	48	48	0	56.7%
Qwen 3.6 Flash	100	100	41	32	0	54.6%
Qwen3.6 Max Preview	100	78	64	26	0	53.7%
DeepSeek V4 Pro (Reasoning)	100	100	65	0	0	52.9%
Claude Opus 4.5	90	86	56	32	0	52.8%
GPT-4o, Aug. 6th (temp=1)	100	100	60	0	0	51.9%
DeepSeek V3 (2024-12-26)	100	94	56	0	0	50.2%
ByteDance Seed 1.6 Flash	100	60	51	35	0	49.0%
Mistral Large 2	100	100	42	0	0	48.3%
GPT-5 Mini	66	55	53	44	21	47.7%
Claude Opus 4.6	81	60	56	34	0	46.2%
Z.AI GLM 4.5	100	85	42	0	0	45.3%
Hermes 3 70B	100	67	57	0	0	44.8%
Gemma 4 31B (Reasoning)	90	46	44	41	0	44.3%
GPT-5	91	45	43	24	16	43.7%
Llama 3.1 70B	72	71	71	0	0	42.9%
Aion 2.0	100	82	25	0	0	41.4%
DeepSeek V4 Pro	92	47	33	29	0	40.1%
Gemini 3.1 Flash Lite	100	48	48	0	0	39.0%
Qwen 3.5 Flash	68	35	35	28	24	38.2%
Claude Opus 4.6 (Reasoning)	68	45	42	33	0	37.6%
Gemma 4 31B	100	48	38	0	0	37.1%
DeepSeek V3.1	77	48	31	28	0	36.8%
Gemini 3 Flash (Preview)	81	65	35	0	0	36.3%
Qwen 3.5 Plus (2026-02-15)	65	39	38	37	0	35.7%
Nemotron 3 Nano	100	40	36	0	0	35.2%
Gemini 3.1 Pro (Preview)	57	55	32	29	0	34.7%
Cydonia 24B V4.1	93	51	29	0	0	34.7%
Gemma 3 12B	100	73	0	0	0	34.7%
Gemini 3 Flash (Preview, Reasoning)	76	62	35	0	0	34.5%
Xiaomi MIMO v2.5	88	56	28	0	0	34.3%
Qwen 3.5 Plus (2026-04-20)	85	59	26	0	0	34.0%
Qwen 3.5 35B	77	54	35	0	0	33.2%
Qwen3.7 Max	100	33	30	0	0	32.6%
Claude Sonnet 4	60	56	48	0	0	32.5%
Hermes 3 405B	85	72	0	0	0	31.6%
GPT-4o Mini (temp=1)	56	53	46	0	0	31.0%
o4 Mini	68	52	33	0	0	30.6%
Qwen 2.5 72B	82	35	31	0	0	29.7%
Gemma 4 26B	40	37	37	33	0	29.5%
Z.AI GLM 4.7	49	48	25	24	0	29.3%
Arcee AI: Trinity Mini	79	67	0	0	0	29.2%
Qwen 3.5 27B	100	32	0	0	0	26.4%
Z.AI GLM 4.6	37	35	32	27	0	26.2%
GPT-5 Nano	74	26	14	13	0	25.4%
Qwen 3.6 35B	69	28	25	0	0	24.6%
ByteDance Seed 2.0 Mini	62	46	0	0	0	21.6%
Qwen 3.5 122B	38	37	28	0	0	20.6%
GPT-4o, Aug. 6th (temp=0)	54	48	0	0	0	20.4%
Grok 4.3 (Reasoning)	100	0	0	0	0	20.0%
WizardLM 2 8x22b	100	0	0	0	0	20.0%
GPT-4o Mini (temp=0)	51	42	0	0	0	18.5%
Gemini 3.1 Flash Lite (Reasoning)	44	41	0	0	0	17.0%
Z.AI GLM 4.5 Air	43	35	0	0	0	15.6%
ByteDance Seed 2.0 Lite	78	0	0	0	0	15.5%
Qwen 3.5 9B	38	25	0	0	0	12.6%
GPT-OSS 120B	29	24	0	0	0	10.5%
Gemini 2.5 Pro	52	0	0	0	0	10.5%
Gemini 3.1 Flash Lite (Preview)	50	0	0	0	0	10.0%
Cohere Command R+ (Aug. 2024)	46	0	0	0	0	9.1%
Qwen 3 32B	26	0	0	0	0	5.2%
Mistral Small 3.2 24B	6	3	0	0	0	1.9%
Inception Mercury 2	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4	100	100	100	100	91	98.1%
Claude Sonnet 5 (Reasoning)	100	100	100	100	90	98.0%
GPT-5.4 Mini	100	100	100	96	90	97.1%
Writer: Palmyra X5	100	100	100	100	72	94.3%
GPT-5.4 (Reasoning, Low)	100	100	100	88	68	91.2%
Claude Opus 4.7 (Reasoning)	100	100	100	97	60	91.1%
GPT-4.1 Nano	100	100	100	100	54	90.8%
GPT-5.4 (Reasoning)	100	100	100	100	50	90.1%
Z.AI GLM 5.2 (Reasoning, High)	100	100	100	100	50	90.0%
DeepSeek V4 Pro	100	100	99	68	53	84.0%
GPT-5.5 (Reasoning, Low)	100	99	82	70	67	83.7%
Qwen3 235B A22B Instruct 2507	100	93	90	88	45	83.4%
Grok 4.5 (Reasoning, High)	100	100	100	56	47	80.5%
Mistral Small 4 (Reasoning)	100	100	100	63	38	80.2%
Grok 4.5 (Reasoning, Low)	100	79	77	74	70	80.1%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	100	0	80.0%
Mistral Small 4	100	100	100	100	0	80.0%
Xiaomi MIMO v2.5 Pro	100	100	83	83	32	79.7%
Claude Opus 4.6 (Reasoning)	100	100	85	73	35	78.7%
Z.AI GLM 5	100	100	100	85	0	77.1%
GPT-5.5	100	100	71	63	46	76.0%
GPT-5.4 Mini (Reasoning)	100	89	68	66	54	75.7%
Claude Opus 4.5	100	100	81	49	42	74.5%
GPT-5.4 Nano (Reasoning, Low)	88	78	73	72	58	73.6%
GPT-5.5 (Reasoning)	98	84	68	56	52	71.7%
Claude Sonnet 5 (Reasoning, Low)	100	85	85	83	0	70.9%
Claude Sonnet 4.5	100	100	88	60	0	69.4%
Gemini 3 Flash (Preview, Reasoning)	100	75	72	58	39	68.9%
MiniMax M2.5	100	100	83	60	0	68.6%
Claude Opus 4	100	90	89	61	0	68.0%
Ministral 3 14B	100	85	58	56	40	67.8%
MiniMax M3	100	100	93	46	0	67.8%
Gemma 3 27B	97	90	52	52	43	66.7%
GPT-5.4 Nano	97	79	54	52	51	66.5%
Claude Sonnet 4	100	100	68	64	0	66.4%
Claude Opus 4.6	100	100	67	59	0	65.3%
GPT-5.4 Mini (Reasoning, Low)	100	91	60	46	28	65.0%
GPT-5.1	100	100	47	40	39	64.9%
Ministral 3 3B	93	88	72	68	0	64.2%
Mistral Large 3	100	100	47	42	30	63.7%
Gemini 3.5 Flash (Reasoning, Minimal)	100	100	65	52	0	63.5%
GPT-4.1 Mini	100	94	63	56	0	62.7%
o4 Mini High	100	84	57	46	26	62.6%
Mistral Large 2	100	100	76	35	0	62.2%
Claude Sonnet 4.6 (Reasoning)	98	67	56	48	43	62.1%
GPT-5.4 Nano (Reasoning)	83	72	67	55	28	61.0%
Gemini 2.5 Flash Lite	95	58	56	52	37	59.8%
GPT-4o Mini (temp=1)	100	95	51	51	0	59.2%
Ministral 8B	100	100	56	40	0	59.2%
Gemma 3 4B	100	53	50	48	42	58.5%
Claude Sonnet 5	100	98	93	0	0	58.1%
WizardLM 2 8x22b	100	89	61	34	0	56.7%
Gemma 3 12B	100	100	41	40	0	56.1%
GPT-5.2	100	80	40	34	19	54.6%
MoonshotAI: Kimi K2.5	100	96	48	29	0	54.5%
Qwen 3 32B	100	82	71	18	0	54.2%
Gemini 2.5 Flash	100	72	55	37	0	52.7%
MoonshotAI: Kimi K2.6	93	71	65	35	0	52.7%
Z.AI GLM 4.7 Flash	100	71	65	27	0	52.5%
DeepSeek V3 (2025-03-24)	100	100	60	0	0	51.9%
GPT-5 Mini	74	60	58	42	22	51.0%
Ministral 3 8B	100	100	54	0	0	50.8%
Z.AI GLM 5 Turbo	100	97	56	0	0	50.4%
GPT-4o, Aug. 6th (temp=1)	100	100	51	0	0	50.1%
DeepSeek V3.1	100	51	47	46	0	48.6%
Claude Haiku 4.5	100	88	53	0	0	48.1%
Cydonia 24B V4.1	100	98	41	0	0	47.8%
Xiaomi MIMO v2.5	72	66	43	29	28	47.7%
Gemini 2.5 Flash Lite (Reasoning)	60	54	49	37	34	46.8%
Claude Opus 4.7	100	67	62	0	0	45.7%
DeepSeek V3 (2024-12-26)	100	88	40	0	0	45.5%
Mistral Medium 3.1	100	97	28	0	0	45.1%
Grok 4.20 (Reasoning)	100	48	38	19	17	44.4%
o4 Mini	93	43	30	29	20	43.1%
Gemma 4 31B	91	75	46	0	0	42.4%
Gemini 3.1 Flash Lite (Preview)	85	44	44	37	0	42.2%
Qwen 3.6 35B	100	67	17	14	12	41.9%
Grok 4.20	100	52	36	22	0	41.8%
DeepSeek V4 Pro (Reasoning)	100	61	46	0	0	41.4%
Aion 2.0	72	66	65	0	0	40.8%
Qwen 3.5 Plus (2026-04-20)	81	48	39	34	0	40.5%
Gemini 2.5 Flash (Reasoning)	100	58	41	0	0	39.8%
Z.AI GLM 4.5	100	47	46	0	0	38.6%
Mistral NeMO	100	58	34	0	0	38.6%
Hermes 3 70B	100	48	39	0	0	37.4%
Gemini 3.1 Flash Lite	81	67	38	0	0	37.2%
GPT-5 Nano	80	38	34	17	15	36.6%
Z.AI GLM 5.1	100	53	29	0	0	36.4%
Cohere Command R+ (Aug. 2024)	100	81	0	0	0	36.3%
Qwen 2.5 72B	78	52	47	0	0	35.3%
Gemma 4 26B (Reasoning)	72	56	47	0	0	35.0%
Qwen 3.5 397B A17B	69	46	41	17	0	34.7%
Mistral Small 3.2 24B	95	77	0	0	0	34.3%
Gemma 4 26B	89	44	39	0	0	34.3%
GPT-4.1	100	36	33	0	0	33.8%
MiniMax M2.7	71	53	44	0	0	33.8%
Gemini 3 Flash (Preview)	72	53	43	0	0	33.6%
ByteDance Seed 2.0 Mini	53	43	35	33	0	32.7%
Claude Opus 4.8 (Reasoning)	100	61	0	0	0	32.1%
ByteDance Seed 1.6 Flash	68	31	31	26	0	31.3%
Claude Sonnet 4.6	100	56	0	0	0	31.1%
Grok 4.3	63	55	36	0	0	30.9%
DeepSeek V4 Flash	54	53	46	0	0	30.6%
Qwen 3.5 Plus (2026-02-15)	45	37	36	34	0	30.5%
Ministral 3B	83	68	0	0	0	30.3%
Gemini 3.5 Flash (Reasoning)	80	34	32	0	0	29.2%
Gemini 2.5 Pro	98	44	0	0	0	28.5%
Z.AI GLM 4.6	53	47	40	0	0	27.9%
Gemini 3.1 Flash Lite (Reasoning)	100	39	0	0	0	27.8%
Z.AI GLM 4.7	70	38	27	0	0	27.0%
DeepSeek V4 Flash (Reasoning)	51	41	39	0	0	26.1%
Aion 3.0	49	38	37	0	0	24.8%
Qwen 3.5 9B	100	23	0	0	0	24.7%
Gemini 3.1 Pro (Preview)	45	27	26	25	0	24.5%
Qwen 3.5 35B	69	33	19	0	0	24.3%
Qwen 3.6 Flash	92	27	0	0	0	23.7%
GPT-5	56	33	27	0	0	23.3%
Grok 4.3 (Reasoning)	60	56	0	0	0	23.0%
Qwen 3.5 122B	72	30	13	0	0	22.9%
Qwen 3.6 27B	56	23	17	16	0	22.5%
GPT-4o, Aug. 6th (temp=0)	62	51	0	0	0	22.4%
Qwen 3.5 Flash	48	45	9	0	0	20.4%
Qwen3.7 Max	65	37	0	0	0	20.4%
Hermes 3 405B	100	0	0	0	0	20.0%
GPT-4o Mini (temp=0)	85	0	0	0	0	17.1%
DeepSeek V3.2	43	39	0	0	0	16.4%
Qwen3.6 Max Preview	39	19	19	0	0	15.4%
Qwen 3.5 27B	30	26	12	0	0	13.6%
DeepSeek-V2 Chat	51	0	0	0	0	10.3%
ByteDance Seed 2.0 Lite	50	0	0	0	0	10.0%
Gemma 4 31B (Reasoning)	48	0	0	0	0	9.5%
Aion 3.0 Mini	44	0	0	0	0	8.9%
Nemotron 3 Super	39	0	0	0	0	7.8%
Z.AI GLM 4.5 Air	36	0	0	0	0	7.2%
Nemotron 3 Nano	35	0	0	0	0	7.0%
ByteDance Seed 1.6	32	0	0	0	0	6.4%
GPT-OSS 120B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.5 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.5 (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Sonnet 5	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	99	99.8%
Arcee AI: Trinity Mini	100	100	100	100	98	99.6%
Grok 4.5 (Reasoning, High)	100	100	100	100	97	99.5%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	94	98.8%
Mistral Small 4	100	100	100	99	94	98.7%
Gemini 2.5 Flash Lite	100	100	100	100	89	97.8%
Claude Opus 4	100	100	100	100	87	97.3%
Z.AI GLM 5	100	100	100	100	86	97.2%
Grok 4.5 (Reasoning, Low)	100	100	100	100	81	96.3%
GPT-5.5	100	100	100	93	84	95.4%
Gemma 3 27B	100	100	100	100	77	95.3%
Grok 4.20	100	100	100	100	75	95.1%
Mistral Small 4 (Reasoning)	100	100	100	100	62	92.5%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	100	62	92.3%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	61	92.3%
GPT-5.1	100	100	100	96	64	92.1%
Claude Opus 4.5	100	100	100	100	55	91.0%
Claude Sonnet 5 (Reasoning)	100	100	100	100	55	90.9%
Ministral 3 8B	100	100	100	87	67	90.6%
Mistral Large 2	100	100	100	100	48	89.7%
Ministral 8B	100	100	100	100	48	89.5%
Claude Opus 4.6	100	100	100	100	46	89.3%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	44	88.9%
WizardLM 2 8x22b	100	100	98	90	56	88.8%
Grok 4.20 (Reasoning)	100	100	100	86	57	88.7%
GPT-5.4 Mini (Reasoning)	100	100	100	100	43	88.6%
Z.AI GLM 4.6	100	100	93	89	60	88.4%
DeepSeek V3 (2024-12-26)	100	100	93	81	68	88.4%
Cydonia 24B V4.1	100	100	100	100	41	88.1%
Claude Opus 4.8 (Reasoning, Low)	100	100	100	93	47	87.9%
Gemini 3.5 Flash (Reasoning, Minimal)	100	95	95	81	67	87.8%
GPT-5.4 Nano (Reasoning)	100	100	100	83	56	87.7%
Mistral Large 3	100	100	100	100	38	87.7%
GPT-5.2	100	100	92	92	51	86.9%
Z.AI GLM 4.7	100	100	92	87	56	86.8%
Z.AI GLM 5.2 (Reasoning, High)	100	100	100	100	33	86.7%
Z.AI GLM 5 Turbo	100	100	100	100	32	86.5%
MiniMax M2.5	100	95	91	88	55	86.0%
Xiaomi MIMO v2.5 Pro	100	100	89	88	48	85.1%
Hermes 3 70B	100	100	100	64	56	84.1%
Z.AI GLM 4.5 Air	100	100	100	60	60	83.9%
Claude Opus 4.7 (Reasoning)	100	100	100	78	41	83.9%
Claude Opus 4.7	100	100	100	78	38	83.3%
Qwen 3.6 Flash	100	100	95	70	50	83.1%
DeepSeek V3.1	100	100	100	58	57	83.1%
GPT-4.1	100	98	75	70	67	82.0%
Qwen 3.5 Plus (2026-04-20)	100	100	98	70	40	81.7%
Claude Sonnet 4	100	100	100	58	50	81.6%
DeepSeek V4 Flash (Reasoning)	100	99	90	78	37	80.9%
DeepSeek V3.2	100	100	95	57	49	80.4%
Z.AI GLM 4.5	100	100	100	100	0	80.0%
DeepSeek-V2 Chat	100	100	100	100	0	80.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	0	80.0%
Ministral 3 14B	100	100	100	100	0	80.0%
Mistral NeMO	100	100	100	100	0	80.0%
GPT-4o Mini (temp=0)	100	100	83	78	39	79.9%
DeepSeek V4 Flash	100	100	80	69	49	79.7%
Claude Sonnet 4.6	100	100	100	98	0	79.6%
DeepSeek V4 Pro	100	100	100	71	27	79.6%
Gemini 2.5 Flash	100	84	77	73	63	79.6%
GPT-4.1 Mini	100	100	100	57	40	79.5%
Claude Opus 4.6 (Reasoning)	100	100	100	72	24	79.2%
GPT-5.4 Nano (Reasoning, Low)	100	94	82	67	51	78.7%
Claude Haiku 4.5	100	100	72	63	57	78.6%
Claude Opus 4.8 (Reasoning)	100	100	100	46	45	78.3%
o4 Mini High	100	100	100	85	0	77.1%
o4 Mini	100	100	80	59	45	76.7%
Aion 3.0	100	100	100	52	30	76.6%
Z.AI GLM 5.1	100	100	100	55	28	76.5%
Gemini 2.5 Pro	100	100	92	80	0	74.4%
Z.AI GLM 4.7 Flash	100	78	72	65	55	74.1%
Gemini 3.1 Flash Lite (Reasoning)	100	82	77	67	39	73.1%
MiniMax M2.7	100	100	85	80	0	73.1%
Qwen3.6 Max Preview	100	96	65	63	41	72.9%
MiniMax M3	100	100	73	53	36	72.6%
Ministral 3B	100	100	100	61	0	72.1%
DeepSeek V4 Pro (Reasoning)	100	99	83	57	22	72.1%
Llama 3.1 70B	100	93	93	71	0	71.2%
ByteDance Seed 1.6 Flash	100	91	87	74	0	70.3%
Nemotron 3 Super	100	95	94	54	0	68.6%
GPT-4o Mini (temp=1)	100	100	100	41	0	68.1%
Aion 3.0 Mini	100	100	74	32	31	67.3%
Gemma 4 31B	100	93	73	36	30	66.7%
Aion 2.0	100	77	67	64	19	65.3%
Xiaomi MIMO v2.5	85	75	65	53	48	65.2%
Hermes 3 405B	100	100	78	43	0	64.1%
Gemini 2.5 Flash Lite (Reasoning)	100	79	65	41	18	60.5%
Gemini 3.1 Flash Lite (Preview)	100	100	100	0	0	60.0%
GPT-5 Mini	100	82	71	39	0	58.4%
Qwen 3.5 Plus (2026-02-15)	100	76	72	44	0	58.4%
Qwen 3.6 27B	100	80	55	45	12	58.3%
MoonshotAI: Kimi K2.6	100	66	61	58	0	57.2%
Grok 4.3	100	82	45	28	28	56.8%
Ministral 3 3B	100	93	81	0	0	54.8%
Qwen3.7 Max	100	97	77	0	0	54.7%
Qwen 3.5 Flash	93	64	62	49	0	53.6%
Qwen 3.5 9B	100	100	63	5	0	53.6%
Gemini 3.5 Flash (Reasoning)	100	100	34	25	0	51.8%
Qwen 3 32B	84	82	63	29	0	51.8%
Mistral Small 3.2 24B	100	100	41	8	3	50.4%
Qwen 3.5 397B A17B	75	69	67	23	14	49.6%
Qwen 3.5 35B	71	69	40	37	26	48.8%
Gemini 3 Flash (Preview, Reasoning)	100	46	33	31	24	46.7%
Qwen 3.5 122B	88	77	37	31	0	46.3%
Gemini 3.1 Pro (Preview)	87	51	41	25	25	45.6%
Gemma 4 26B (Reasoning)	80	69	44	32	0	45.0%
Gemini 3.1 Flash Lite	100	69	46	0	0	43.1%
Gemma 4 26B	69	57	49	34	0	41.9%
Qwen 3.6 35B	69	60	38	28	12	41.3%
ByteDance Seed 2.0 Lite	67	57	39	39	0	40.6%
Qwen 2.5 72B	100	46	43	0	0	37.7%
GPT-5 Nano	69	41	34	22	17	36.5%
GPT-4o, Aug. 6th (temp=0)	100	44	38	0	0	36.3%
Grok 4.3 (Reasoning)	71	53	40	13	0	35.3%
GPT-5	88	43	17	13	0	32.3%
Inception Mercury 2	40	34	32	30	0	27.1%
MoonshotAI: Kimi K2.5	58	32	28	0	0	23.8%
Gemini 3 Flash (Preview)	53	27	22	0	0	20.5%
Nemotron 3 Nano	100	0	0	0	0	20.0%
Gemma 4 31B (Reasoning)	31	31	30	0	0	18.5%
Qwen 3.5 27B	53	31	0	0	0	16.9%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	89	97.8%
Ministral 3 8B	100	100	100	95	82	95.5%
Ministral 3 14B	100	100	100	100	78	95.5%
Claude Opus 4.7	100	100	100	100	76	95.2%
Grok 4.20 (Reasoning)	100	100	92	92	81	93.1%
Mistral Small 4	100	100	100	100	62	92.5%
Mistral Small 4 (Reasoning)	100	100	100	100	60	92.0%
Mistral Large 2	100	100	100	100	48	89.5%
Mistral Medium 3.1	100	100	89	83	72	88.8%
Writer: Palmyra X5	100	100	100	97	37	86.7%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	65	62	85.5%
GPT-5.5 (Reasoning)	100	100	85	77	52	82.6%
Claude Opus 4.8 (Reasoning, Low)	100	100	74	68	64	81.2%
Mistral Large 3	100	100	100	100	0	80.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	0	80.0%
Gemma 3 12B	100	94	85	83	37	79.9%
Claude Sonnet 4.6 (Reasoning)	100	100	78	61	41	75.9%
Mistral NeMO	100	100	100	67	0	73.3%
Z.AI GLM 5	100	100	83	46	34	72.6%
Claude Sonnet 4.6	100	100	100	61	0	72.1%
GPT-5.4 Mini	94	78	67	66	52	71.3%
Claude Sonnet 5 (Reasoning)	100	100	83	72	0	71.2%
o4 Mini High	100	100	65	50	35	69.9%
GPT-5.5 (Reasoning, Low)	95	88	72	60	31	69.2%
GPT-5.4	92	85	79	52	37	69.2%
Grok 4.20	100	100	74	67	0	68.1%
Qwen 3 32B	100	100	81	53	0	66.8%
GPT-5.4 Nano	98	87	55	47	41	65.6%
DeepSeek V4 Pro	100	83	61	44	32	64.0%
Gemma 3 27B	100	90	85	37	0	62.6%
Claude Opus 4.5	100	74	67	39	30	62.0%
Claude Sonnet 4.5	100	100	61	48	0	61.6%
Z.AI GLM 5.2 (Reasoning, High)	100	100	59	43	0	60.3%
Ministral 3 3B	100	100	41	29	28	59.7%
Aion 3.0 Mini	100	100	93	0	0	58.7%
Claude Opus 4.7 (Reasoning)	100	100	90	0	0	58.0%
DeepSeek-V2 Chat	100	100	89	0	0	57.8%
GPT-5.4 (Reasoning, Low)	85	84	58	42	19	57.5%
Claude Sonnet 4	100	89	51	42	0	56.3%
Aion 3.0	87	70	69	55	0	56.0%
Claude Opus 4.8 (Reasoning)	79	71	65	63	0	55.7%
MiniMax M2.5	100	100	78	0	0	55.5%
Xiaomi MIMO v2.5 Pro	100	67	46	38	25	55.1%
GPT-5.4 Mini (Reasoning, Low)	84	74	51	42	23	54.8%
Claude Sonnet 5	100	88	85	0	0	54.6%
DeepSeek V3 (2025-03-24)	100	56	56	55	0	53.2%
GPT-5.5	88	66	52	30	27	52.7%
MoonshotAI: Kimi K2.5	100	100	62	0	0	52.5%
Grok 4.5 (Reasoning, High)	100	100	60	0	0	51.9%
Claude Opus 4.6 (Reasoning)	100	69	61	29	0	51.8%
Z.AI GLM 5.1	100	82	40	34	0	51.1%
ByteDance Seed 2.0 Mini	88	63	57	46	0	50.9%
GPT-4.1	100	56	51	46	0	50.8%
ByteDance Seed 1.6 Flash	100	100	41	0	0	48.2%
Ministral 8B	100	48	48	44	0	48.1%
GPT-4.1 Nano	67	64	58	51	0	48.0%
Claude Haiku 4.5	100	51	46	38	0	47.2%
Z.AI GLM 4.6	100	100	35	0	0	46.9%
MoonshotAI: Kimi K2.6	93	51	46	42	0	46.4%
Gemma 3 4B	100	89	40	0	0	45.8%
GPT-5.4 Nano (Reasoning)	67	57	42	34	27	45.5%
MiniMax M3	100	78	48	0	0	45.3%
Xiaomi MIMO v2.5	98	71	30	25	0	44.8%
GPT-5.4 (Reasoning)	97	55	26	24	21	44.6%
Ministral 3B	100	80	41	0	0	44.3%
Qwen 2.5 72B	100	61	58	0	0	43.8%
DeepSeek V4 Pro (Reasoning)	84	70	64	0	0	43.7%
Gemini 3.1 Flash Lite (Reasoning)	100	78	38	0	0	43.1%
Z.AI GLM 4.5	98	67	50	0	0	42.9%
DeepSeek V4 Flash (Reasoning)	79	48	47	39	0	42.7%
MiniMax M2.7	94	78	33	0	0	40.9%
GPT-5.4 Mini (Reasoning)	73	49	31	29	20	40.5%
Claude Sonnet 5 (Reasoning, Low)	100	100	0	0	0	40.0%
GPT-4o Mini (temp=1)	100	52	44	0	0	39.3%
Aion 2.0	100	61	35	0	0	39.2%
Cydonia 24B V4.1	71	61	60	0	0	38.2%
o4 Mini	100	61	26	0	0	37.5%
Z.AI GLM 5 Turbo	100	80	0	0	0	36.1%
GPT-4o, Aug. 6th (temp=1)	65	56	55	0	0	35.3%
GPT-5.1	57	52	39	25	0	34.6%
Hermes 3 70B	100	69	0	0	0	33.9%
DeepSeek V4 Flash	100	68	0	0	0	33.6%
Claude Opus 4.6	100	39	28	0	0	33.4%
ByteDance Seed 2.0 Lite	89	72	0	0	0	32.3%
Gemini 2.5 Flash Lite (Reasoning)	46	42	40	32	0	32.0%
DeepSeek V3.2	93	34	31	0	0	31.5%
Qwen 3.5 35B	45	42	36	29	0	30.5%
Gemma 4 31B (Reasoning)	51	49	49	0	0	29.9%
GPT-5.2	56	53	22	15	0	29.2%
Qwen 3.5 9B	64	56	26	0	0	29.2%
Gemini 2.5 Flash Lite	100	46	0	0	0	29.1%
WizardLM 2 8x22b	68	40	35	0	0	28.7%
Gemini 3.5 Flash (Reasoning, Minimal)	88	54	0	0	0	28.3%
Gemini 3.1 Flash Lite	79	58	0	0	0	27.6%
Claude Opus 4	48	46	42	0	0	27.2%
Qwen 3.5 Flash	52	42	22	19	0	27.0%
GPT-5	40	38	31	22	0	26.1%
Qwen 3.6 Flash	61	46	22	0	0	25.8%
DeepSeek V3.1	79	48	0	0	0	25.4%
Gemini 3.5 Flash (Reasoning)	44	41	40	0	0	25.1%
GPT-5 Nano	67	30	24	0	0	24.1%
Qwen 3.5 397B A17B	46	31	14	13	0	20.6%
Gemini 2.5 Flash	100	0	0	0	0	20.0%
Cohere Command R+ (Aug. 2024)	100	0	0	0	0	20.0%
Qwen 3.6 35B	36	26	20	17	0	19.7%
Gemma 4 26B (Reasoning)	52	45	0	0	0	19.4%
Grok 4.5 (Reasoning, Low)	93	0	0	0	0	18.7%
Grok 4.3	91	0	0	0	0	18.2%
Llama 3.1 70B	85	0	0	0	0	17.1%
Qwen3.6 Max Preview	37	19	16	13	0	16.8%
Qwen 3.5 Plus (2026-04-20)	68	15	0	0	0	16.6%
Qwen 3.5 27B	47	23	13	0	0	16.6%
Gemini 2.5 Pro	79	0	0	0	0	15.9%
Hermes 3 405B	78	0	0	0	0	15.5%
Gemini 3 Flash (Preview)	40	36	0	0	0	15.2%
GPT-5 Mini	28	26	22	0	0	15.2%
Qwen 3.5 122B	53	23	0	0	0	15.0%
Z.AI GLM 4.7	39	36	0	0	0	14.9%
Z.AI GLM 4.5 Air	37	36	0	0	0	14.7%
Grok 4.3 (Reasoning)	64	0	0	0	0	12.8%
Z.AI GLM 4.7 Flash	62	0	0	0	0	12.3%
GPT-4.1 Mini	62	0	0	0	0	12.3%
GPT-4o Mini (temp=0)	53	0	0	0	0	10.6%
Gemma 4 26B	44	0	0	0	0	8.9%
Inception Mercury 2	44	0	0	0	0	8.9%
Qwen 3.6 27B	29	15	0	0	0	8.7%
ByteDance Seed 1.6	36	0	0	0	0	7.2%
Gemini 3.1 Flash Lite (Preview)	36	0	0	0	0	7.2%
Gemini 2.5 Flash (Reasoning)	33	0	0	0	0	6.7%
Gemini 3 Flash (Preview, Reasoning)	32	0	0	0	0	6.3%
Qwen3.7 Max	22	0	0	0	0	4.4%
Mistral Small 3.2 24B	20	0	0	0	0	4.0%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
Qwen 3.5 Plus (2026-02-15)	0	0	0	0	0	0.0%
Gemma 4 31B	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	89	97.8%
Mistral Medium 3.1	100	100	100	100	88	97.5%
Claude Sonnet 5	100	100	88	79	72	87.9%
Writer: Palmyra X5	100	100	100	100	0	80.0%
Ministral 8B	100	100	100	94	0	78.8%
Ministral 3B	100	100	100	76	0	75.2%
Ministral 3 8B	100	100	100	65	0	73.1%
MiniMax M2.7	100	100	83	46	35	72.7%
DeepSeek V3 (2025-03-24)	100	100	95	68	0	72.7%
MiniMax M3	100	100	79	48	36	72.6%
Gemma 4 26B	98	85	72	58	48	72.4%
Cohere Command R+ (Aug. 2024)	100	100	100	50	0	70.0%
Z.AI GLM 5.1	100	82	80	75	0	67.6%
Mistral Large 2	100	100	100	36	0	67.2%
Gemini 2.5 Flash Lite (Reasoning)	95	81	55	51	51	66.7%
Claude Haiku 4.5	100	93	54	51	35	66.4%
GPT-5.4	100	100	61	42	22	65.0%
Xiaomi MIMO v2.5	100	100	89	30	0	63.8%
Ministral 3 14B	100	100	60	52	0	62.3%
Claude Opus 4.6	100	84	71	55	0	62.0%
Mistral Small 4	100	81	74	55	0	62.0%
Gemini 3.1 Flash Lite (Reasoning)	100	87	65	55	0	61.2%
Mistral Large 3	100	100	100	0	0	60.0%
GPT-4.1 Nano	100	100	100	0	0	60.0%
Aion 3.0	100	100	66	32	0	59.6%
Qwen 3 32B	100	100	97	0	0	59.5%
GPT-5.4 (Reasoning)	100	79	63	48	0	57.9%
Gemma 3 27B	100	85	53	38	0	55.3%
Qwen 3.5 Plus (2026-02-15)	100	69	41	35	28	54.6%
GPT-5.4 Nano (Reasoning, Low)	72	66	56	53	25	54.5%
GPT-5.1	69	68	59	53	21	54.0%
Claude Opus 4.6 (Reasoning)	100	88	53	28	0	53.7%
Gemini 3.5 Flash (Reasoning, Minimal)	97	90	80	0	0	53.4%
GPT-5.4 Mini	100	64	57	24	21	53.0%
ByteDance Seed 2.0 Lite	100	84	47	34	0	53.0%
Qwen 3.5 122B	99	84	52	17	13	52.9%
Qwen 3.6 35B	100	74	58	17	15	52.7%
Hermes 3 405B	100	81	67	0	0	49.6%
GPT-5.4 Nano	59	58	51	45	31	48.6%
Qwen 3.6 Flash	100	50	45	45	0	48.0%
DeepSeek V4 Pro (Reasoning)	82	65	51	41	0	47.6%
Grok 4.20 (Reasoning)	100	50	43	24	20	47.3%
GPT-4.1	100	100	36	0	0	47.2%
Gemini 3.1 Flash Lite (Preview)	67	50	43	43	31	46.7%
ByteDance Seed 1.6 Flash	100	84	47	0	0	46.2%
GPT-4.1 Mini	100	78	51	0	0	45.8%
Aion 3.0 Mini	80	63	43	42	0	45.8%
o4 Mini High	88	83	32	23	0	45.5%
Claude Opus 4.5	100	67	60	0	0	45.4%
Gemini 3.1 Flash Lite	100	78	46	0	0	44.8%
GPT-5.5	84	64	43	17	15	44.7%
DeepSeek V3 (2024-12-26)	100	79	44	0	0	44.6%
GPT-4o Mini (temp=1)	100	71	49	0	0	44.0%
DeepSeek V4 Flash (Reasoning)	100	72	47	0	0	43.9%
GPT-5.4 Mini (Reasoning)	100	72	24	20	0	43.2%
Gemma 3 4B	99	78	35	0	0	42.3%
Z.AI GLM 4.7 Flash	100	71	41	0	0	42.3%
MoonshotAI: Kimi K2.5	100	66	45	0	0	42.2%
MoonshotAI: Kimi K2.6	100	75	36	0	0	42.1%
Qwen3.6 Max Preview	74	50	31	30	26	42.1%
Claude Opus 4.7 (Reasoning)	95	62	48	0	0	40.9%
Gemini 3.5 Flash (Reasoning)	76	46	43	40	0	40.9%
GPT-5.5 (Reasoning, Low)	86	60	43	15	0	40.7%
Gemma 4 26B (Reasoning)	100	54	50	0	0	40.6%
Gemini 2.5 Flash (Reasoning)	91	75	36	0	0	40.4%
Grok 4.20	71	56	35	20	19	40.1%
Claude Opus 4.7	100	100	0	0	0	40.0%
Hermes 3 70B	100	100	0	0	0	40.0%
Claude Opus 4.8 (Reasoning)	100	97	0	0	0	39.3%
GPT-5.4 Nano (Reasoning)	84	50	27	24	12	39.3%
Qwen 3.5 35B	100	46	34	15	0	39.0%
Qwen 3.5 397B A17B	92	71	31	0	0	38.6%
GPT-5.4 (Reasoning, Low)	87	39	23	22	21	38.3%
o4 Mini	88	52	51	0	0	38.1%
Gemini 3 Flash (Preview)	64	56	37	32	0	37.9%
DeepSeek V4 Flash	51	47	46	42	0	37.1%
Claude Sonnet 5 (Reasoning, Low)	100	81	0	0	0	36.3%
DeepSeek V3.1	94	83	0	0	0	35.4%
Qwen 3.5 Flash	53	49	41	34	0	35.3%
Gemini 2.5 Flash Lite	100	75	0	0	0	35.0%
Claude Sonnet 5 (Reasoning)	95	76	0	0	0	34.2%
Gemma 3 12B	82	48	40	0	0	34.0%
Claude Sonnet 4.5	85	51	32	0	0	33.6%
Cydonia 24B V4.1	94	72	0	0	0	33.3%
Mistral NeMO	63	58	44	0	0	33.2%
GPT-4o, Aug. 6th (temp=1)	100	64	0	0	0	32.8%
Z.AI GLM 5	100	34	29	0	0	32.6%
GPT-5.4 Mini (Reasoning, Low)	92	25	23	21	0	32.2%
Claude Opus 4	66	48	45	0	0	31.9%
Claude Opus 4.8 (Reasoning, Low)	55	53	49	0	0	31.3%
WizardLM 2 8x22b	64	37	28	27	0	31.1%
Z.AI GLM 5 Turbo	55	54	44	0	0	30.6%
Gemma 4 31B	100	51	0	0	0	30.3%
GPT-4o, Aug. 6th (temp=0)	100	49	0	0	0	29.8%
GPT-5	52	39	36	23	0	29.7%
GPT-5 Nano	56	49	38	0	0	28.7%
ByteDance Seed 2.0 Mini	100	42	0	0	0	28.4%
Z.AI GLM 4.5 Air	78	64	0	0	0	28.3%
Z.AI GLM 5.2 (Reasoning, High)	100	39	0	0	0	27.8%
Grok 4.5 (Reasoning, High)	63	41	33	0	0	27.3%
Ministral 3 3B	71	65	0	0	0	27.3%
Qwen 3.6 27B	44	39	34	19	0	27.2%
DeepSeek V4 Pro	90	45	0	0	0	27.0%
Qwen 3.5 Plus (2026-04-20)	94	26	15	0	0	27.0%
DeepSeek V3.2	64	38	32	0	0	26.8%
GPT-5.5 (Reasoning)	34	34	33	29	0	26.0%
Grok 4.5 (Reasoning, Low)	54	46	30	0	0	25.9%
Gemini 2.5 Pro	79	49	0	0	0	25.7%
Aion 2.0	87	41	0	0	0	25.4%
Gemini 2.5 Flash	92	35	0	0	0	25.4%
Gemini 3 Flash (Preview, Reasoning)	89	29	0	0	0	23.6%
Z.AI GLM 4.6	39	39	38	0	0	23.3%
Xiaomi MIMO v2.5 Pro	76	40	0	0	0	23.2%
Qwen 2.5 72B	71	41	0	0	0	22.4%
Claude Sonnet 4.6	62	49	0	0	0	22.1%
GPT-5.2	34	27	22	19	0	20.3%
Llama 3.1 70B	100	0	0	0	0	20.0%
DeepSeek-V2 Chat	53	46	0	0	0	19.7%
Claude Sonnet 4.6 (Reasoning)	51	45	0	0	0	19.1%
Gemma 4 31B (Reasoning)	46	44	0	0	0	18.0%
Arcee AI: Trinity Mini	90	0	0	0	0	18.0%
GPT-5 Mini	39	25	19	0	0	16.6%
Grok 4.3 (Reasoning)	81	0	0	0	0	16.1%
ByteDance Seed 1.6	78	0	0	0	0	15.5%
Grok 4.3	43	31	0	0	0	14.7%
Qwen 3.5 9B	45	28	0	0	0	14.6%
Mistral Small 3.2 24B	68	1	0	0	0	13.9%
Claude Sonnet 4	58	0	0	0	0	11.7%
MiniMax M2.5	58	0	0	0	0	11.7%
Z.AI GLM 4.5	50	0	0	0	0	10.0%
GPT-4o Mini (temp=0)	48	0	0	0	0	9.7%
GPT-OSS 120B	30	0	0	0	0	6.0%
Qwen 3.5 27B	19	0	0	0	0	3.8%
Qwen3.7 Max	0	0	0	0	0	0.0%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
Z.AI GLM 4.7	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 Mini	100	100	100	100	87	97.4%
GPT-5.4 (Reasoning, Low)	100	100	100	90	83	94.5%
Grok 4.20	100	100	100	100	70	94.0%
Mistral Medium 3.1	100	100	100	89	78	93.3%
Qwen 3 32B	100	100	100	100	64	92.8%
MiniMax M2.5	100	100	100	97	67	92.7%
Writer: Palmyra X5	100	100	100	100	57	91.5%
Mistral Small 4 (Reasoning)	100	100	100	76	68	89.0%
GPT-5.4 Mini (Reasoning)	100	100	100	96	47	88.6%
Ministral 3 14B	100	100	100	100	37	87.3%
Claude Opus 4	100	100	100	84	38	84.5%
GPT-5.5 (Reasoning, Low)	100	100	90	84	49	84.5%
Claude Opus 4.6	100	100	100	92	29	84.1%
Claude Opus 4.6 (Reasoning)	100	100	97	80	37	82.7%
GPT-5.5	100	100	91	69	45	81.0%
Ministral 8B	100	100	100	100	0	80.0%
Ministral 3 8B	100	100	82	56	51	77.8%
Gemma 3 27B	100	100	100	85	0	77.0%
Claude Opus 4.5	100	100	98	42	38	75.7%
Hermes 3 405B	100	84	72	61	58	75.1%
Grok 4.20 (Reasoning)	100	90	79	69	36	74.9%
ByteDance Seed 1.6 Flash	100	100	100	40	34	74.7%
GPT-5.4 Mini (Reasoning, Low)	100	98	74	52	44	73.4%
Claude Opus 4.8 (Reasoning, Low)	100	100	95	67	0	72.4%
GPT-5.4 (Reasoning)	100	100	65	57	39	72.3%
Claude Opus 4.7 (Reasoning)	100	100	100	56	0	71.3%
GPT-4o, Aug. 6th (temp=1)	100	100	100	55	0	70.9%
o4 Mini High	100	90	85	56	23	70.9%
GPT-5.4 Nano	86	80	71	60	53	70.1%
MoonshotAI: Kimi K2.5	100	96	73	42	37	69.7%
GPT-5.4	100	100	97	51	0	69.5%
Qwen3 235B A22B Instruct 2507	100	100	100	47	0	69.4%
DeepSeek V4 Flash	100	95	54	52	42	68.7%
Aion 3.0 Mini	91	79	63	62	43	67.6%
GPT-5.5 (Reasoning)	100	87	72	46	29	66.9%
Mistral Large 2	100	71	67	55	42	66.8%
DeepSeek V3 (2025-03-24)	100	95	89	50	0	66.8%
Ministral 3 3B	100	100	79	53	0	66.5%
Xiaomi MIMO v2.5 Pro	100	100	46	43	39	65.5%
GPT-5.1	100	100	85	39	0	64.8%
Xiaomi MIMO v2.5	100	90	58	42	30	64.0%
Claude Haiku 4.5	89	62	61	53	50	62.8%
Cydonia 24B V4.1	100	51	51	48	41	58.0%
Mistral Large 3	100	77	75	37	0	57.7%
Gemini 2.5 Flash Lite (Reasoning)	100	68	52	36	30	57.1%
Gemma 3 12B	100	100	44	38	0	56.5%
GPT-4.1 Nano	100	74	55	53	0	56.3%
Z.AI GLM 5.1	100	100	46	32	0	55.7%
Z.AI GLM 5 Turbo	100	51	46	42	34	54.4%
Qwen 3.6 Flash	79	65	59	51	16	53.8%
GPT-5 Nano	100	64	61	26	17	53.7%
Z.AI GLM 4.6	87	84	54	39	0	52.6%
GPT-5.4 Nano (Reasoning, Low)	100	68	40	38	14	52.2%
GPT-4.1	91	90	74	0	0	51.1%
Aion 3.0	100	78	76	0	0	50.8%
Qwen 3.5 35B	100	95	33	16	6	50.1%
DeepSeek V4 Pro (Reasoning)	100	63	48	35	0	49.4%
Mistral Small 4	100	54	48	43	0	48.9%
GPT-5.2	100	45	44	37	14	48.1%
WizardLM 2 8x22b	100	62	44	34	0	47.9%
Qwen 3.6 35B	98	67	48	25	0	47.8%
Claude Opus 4.8 (Reasoning)	100	76	63	0	0	47.7%
DeepSeek V3.1	100	95	43	0	0	47.6%
Qwen 3.5 397B A17B	100	64	51	19	0	46.7%
Gemini 3.1 Flash Lite (Reasoning)	100	60	37	32	0	46.0%
MiniMax M3	78	58	48	39	0	44.6%
MiniMax M2.7	100	75	43	0	0	43.5%
Z.AI GLM 4.7 Flash	75	51	48	41	0	42.8%
DeepSeek V3 (2024-12-26)	100	65	44	0	0	41.8%
GPT-4o Mini (temp=1)	100	57	48	0	0	41.0%
Ministral 3B	100	57	44	0	0	40.1%
Claude Sonnet 4.6	100	100	0	0	0	40.0%
Claude Sonnet 5 (Reasoning)	100	100	0	0	0	40.0%
Claude Sonnet 5 (Reasoning, Low)	100	100	0	0	0	40.0%
DeepSeek-V2 Chat	100	100	0	0	0	40.0%
Hermes 3 70B	100	100	0	0	0	40.0%
Arcee AI: Trinity Mini	100	100	0	0	0	40.0%
Gemini 3.1 Flash Lite (Preview)	48	46	39	34	31	39.7%
DeepSeek V4 Flash (Reasoning)	61	51	49	37	0	39.5%
Gemini 2.5 Pro	100	97	0	0	0	39.4%
Claude Sonnet 4	76	64	56	0	0	39.3%
Claude Sonnet 4.5	100	51	45	0	0	39.1%
Qwen 3.5 Plus (2026-04-20)	65	46	38	32	12	38.6%
Grok 4.5 (Reasoning, Low)	98	88	0	0	0	37.2%
ByteDance Seed 2.0 Lite	100	85	0	0	0	37.1%
Grok 4.3	72	57	55	0	0	36.9%
Z.AI GLM 5.2 (Reasoning, High)	100	40	35	0	0	35.1%
Z.AI GLM 5	100	42	31	0	0	34.6%
Gemini 3.5 Flash (Reasoning)	73	72	28	0	0	34.6%
Gemini 2.5 Flash Lite	82	59	31	0	0	34.5%
GPT-5	69	43	37	23	0	34.5%
Claude Sonnet 4.6 (Reasoning)	64	60	47	0	0	34.1%
ByteDance Seed 2.0 Mini	85	84	0	0	0	34.0%
Z.AI GLM 4.5	79	48	41	0	0	33.8%
Claude Opus 4.7	100	62	0	0	0	32.3%
Qwen 3.5 122B	59	40	39	24	0	32.3%
Gemma 3 4B	100	55	0	0	0	30.9%
Gemini 3.5 Flash (Reasoning, Minimal)	52	51	46	0	0	29.7%
Z.AI GLM 4.5 Air	100	46	0	0	0	29.1%
Grok 4.5 (Reasoning, High)	72	38	32	0	0	28.6%
DeepSeek V3.2	65	44	31	0	0	28.2%
Qwen 3.5 Flash	100	38	0	0	0	27.5%
Gemini 3.1 Flash Lite	100	37	0	0	0	27.3%
GPT-4.1 Mini	68	67	0	0	0	26.9%
GPT-5.4 Nano (Reasoning)	34	33	26	24	17	26.8%
Gemma 4 26B (Reasoning)	69	63	0	0	0	26.5%
Z.AI GLM 4.7	100	31	0	0	0	26.2%
Gemini 3 Flash (Preview, Reasoning)	60	37	32	0	0	25.8%
DeepSeek V4 Pro	100	28	0	0	0	25.6%
GPT-5 Mini	28	27	26	25	21	25.3%
o4 Mini	91	31	0	0	0	24.4%
Qwen3.6 Max Preview	49	45	25	0	0	23.9%
Qwen 3.5 27B	39	26	23	20	0	21.6%
Claude Sonnet 5	100	0	0	0	0	20.0%
Llama 3.1 70B	100	0	0	0	0	20.0%
Cohere Command R+ (Aug. 2024)	100	0	0	0	0	20.0%
Gemini 3 Flash (Preview)	36	35	29	0	0	20.0%
Qwen 3.5 9B	60	38	0	0	0	19.6%
Nemotron 3 Super	97	0	0	0	0	19.3%
Gemma 4 31B	49	47	0	0	0	19.2%
Qwen 3.5 Plus (2026-02-15)	56	39	0	0	0	18.9%
Aion 2.0	40	30	23	0	0	18.8%
Gemma 4 26B	46	44	0	0	0	18.1%
Gemini 2.5 Flash	47	35	0	0	0	16.4%
Qwen 3.6 27B	49	30	0	0	0	15.7%
Mistral Small 3.2 24B	78	0	0	0	0	15.5%
Qwen3.7 Max	54	23	0	0	0	15.4%
MoonshotAI: Kimi K2.6	35	34	0	0	0	13.8%
Gemini 2.5 Flash (Reasoning)	59	0	0	0	0	11.8%
Qwen 2.5 72B	58	0	0	0	0	11.7%
Gemma 4 31B (Reasoning)	54	0	0	0	0	10.8%
Nemotron 3 Nano	46	0	0	0	0	9.1%
Mistral NeMO	42	0	0	0	0	8.4%
Grok 4.3 (Reasoning)	31	0	0	0	0	6.1%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
ByteDance Seed 1.6	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	89	97.9%
Ministral 3 14B	100	100	100	100	85	97.0%
Z.AI GLM 5 Turbo	100	100	100	100	79	95.9%
Ministral 8B	100	100	100	100	71	94.2%
Mistral Small 4 (Reasoning)	100	100	100	100	52	90.5%
GPT-4.1 Nano	100	100	100	100	51	90.3%
GPT-5.4	100	100	100	99	51	89.8%
Mistral Small 4	100	100	100	100	45	89.0%
Claude Sonnet 4.5	100	93	85	84	78	88.2%
Qwen3 235B A22B Instruct 2507	100	100	100	100	34	86.8%
Claude Opus 4.7 (Reasoning)	100	100	82	74	71	85.5%
GPT-5.4 (Reasoning, Low)	100	100	100	84	39	84.8%
Claude Sonnet 5 (Reasoning, Low)	100	100	100	62	62	84.7%
Gemma 3 4B	100	87	79	72	68	81.3%
Hermes 3 70B	100	100	80	68	56	80.8%
Aion 3.0	100	100	100	53	51	80.7%
ByteDance Seed 1.6 Flash	100	100	95	65	38	79.6%
Claude Sonnet 5 (Reasoning)	100	100	65	64	62	78.2%
Z.AI GLM 5.2 (Reasoning, High)	100	100	100	55	33	77.6%
GPT-5.4 Mini (Reasoning, Low)	100	100	78	61	40	76.0%
Claude Opus 4.8 (Reasoning)	100	100	82	47	46	75.1%
GPT-5.4 (Reasoning)	100	95	60	60	51	73.2%
DeepSeek V4 Pro (Reasoning)	96	85	75	72	36	73.0%
Z.AI GLM 5	100	100	82	81	0	72.7%
Mistral Large 2	100	100	82	79	0	72.3%
GPT-5.4 Mini (Reasoning)	100	100	98	63	0	72.3%
Grok 4.20	100	88	78	68	23	71.3%
MiniMax M2.7	100	100	97	54	0	70.3%
Z.AI GLM 5.1	89	84	76	66	35	70.1%
Claude Haiku 4.5	100	100	83	33	32	69.8%
Mistral Medium 3.1	100	95	74	43	35	69.3%
Claude Sonnet 5	100	67	61	61	58	69.3%
GPT-4o Mini (temp=1)	100	100	98	48	0	69.3%
Mistral Large 3	100	100	100	43	0	68.7%
GPT-5.4 Mini	100	100	88	52	0	68.1%
DeepSeek-V2 Chat	100	98	93	47	0	67.5%
Grok 4.5 (Reasoning, Low)	100	89	70	39	34	66.5%
Ministral 3 8B	100	100	87	39	0	65.2%
Aion 3.0 Mini	100	93	88	43	0	64.8%
MiniMax M3	100	100	83	38	0	64.3%
Claude Sonnet 4	100	100	67	53	0	63.9%
Gemma 3 12B	100	91	58	35	27	62.5%
DeepSeek V4 Pro	93	89	73	30	27	62.4%
Gemini 2.5 Flash (Reasoning)	98	80	50	45	37	62.0%
Grok 4.5 (Reasoning, High)	100	100	79	29	0	61.8%
Gemini 2.5 Flash Lite (Reasoning)	100	100	63	44	0	61.4%
GPT-5.1	100	65	56	55	31	61.3%
Claude Opus 4.6	100	84	55	39	24	60.4%
GPT-5.5 (Reasoning, Low)	100	81	44	41	35	60.2%
GPT-5.5	87	65	57	54	39	60.2%
Ministral 3 3B	100	100	100	0	0	59.9%
GPT-4.1	100	93	71	34	0	59.7%
GPT-4.1 Mini	100	100	98	0	0	59.6%
Xiaomi MIMO v2.5	93	79	68	54	0	58.9%
Grok 4.20 (Reasoning)	79	77	52	52	31	58.0%
Gemini 3.5 Flash (Reasoning, Minimal)	94	84	52	51	0	56.2%
DeepSeek V4 Flash (Reasoning)	100	78	59	40	0	55.5%
GPT-5.4 Nano (Reasoning)	94	85	60	35	0	54.8%
Mistral NeMO	100	100	73	0	0	54.7%
GPT-5.4 Nano	95	88	43	36	11	54.5%
MiniMax M2.5	100	77	65	29	0	54.3%
Z.AI GLM 4.6	100	100	38	30	0	53.7%
Gemma 3 27B	100	100	36	32	0	53.6%
Z.AI GLM 4.7 Flash	100	58	44	34	28	53.0%
GPT-5.5 (Reasoning)	76	59	53	44	33	52.8%
Gemma 4 31B	97	93	36	35	0	52.4%
Gemini 2.5 Flash Lite	100	65	46	46	0	51.5%
Claude Sonnet 4.6	100	100	56	0	0	51.1%
GPT-5.4 Nano (Reasoning, Low)	87	54	51	38	26	51.1%
Claude Sonnet 4.6 (Reasoning)	100	100	54	0	0	50.8%
Claude Opus 4.5	73	63	53	31	29	49.8%
MoonshotAI: Kimi K2.6	79	77	36	31	25	49.7%
DeepSeek V3 (2025-03-24)	100	100	46	0	0	49.3%
Claude Opus 4.6 (Reasoning)	82	62	50	27	23	48.8%
Arcee AI: Trinity Mini	83	60	54	45	0	48.3%
Gemma 4 26B (Reasoning)	93	78	43	26	0	48.0%
Aion 2.0	100	76	64	0	0	48.0%
Qwen 3.5 Flash	78	77	57	23	0	47.1%
Claude Opus 4.8 (Reasoning, Low)	100	47	47	41	0	46.9%
Ministral 3B	100	68	55	0	0	44.5%
DeepSeek V3.1	100	75	45	0	0	44.0%
Cohere Command R+ (Aug. 2024)	91	73	53	0	0	43.5%
Xiaomi MIMO v2.5 Pro	72	63	48	33	0	43.3%
Grok 4.3 (Reasoning)	100	55	30	27	0	42.3%
Qwen 3.5 Plus (2026-02-15)	89	76	23	22	0	42.0%
Cydonia 24B V4.1	100	56	51	0	0	41.3%
Qwen 3.5 397B A17B	71	67	48	20	0	41.1%
Z.AI GLM 4.5 Air	100	100	0	0	0	40.0%
Gemini 3 Flash (Preview)	72	39	38	29	21	39.9%
Nemotron 3 Super	79	65	52	0	0	39.4%
o4 Mini	73	52	31	20	19	39.0%
DeepSeek V3.2	98	34	30	27	0	37.8%
Gemini 2.5 Pro	76	63	38	0	0	35.4%
GPT-4o Mini (temp=0)	91	76	0	0	0	33.4%
MoonshotAI: Kimi K2.5	100	66	0	0	0	33.2%
DeepSeek V3 (2024-12-26)	100	34	26	0	0	32.0%
Gemini 3.1 Flash Lite	100	60	0	0	0	31.8%
GPT-5 Mini	73	38	24	23	0	31.6%
Z.AI GLM 4.7	75	29	26	24	0	31.0%
Qwen 3.5 9B	100	44	9	0	0	30.5%
Qwen 3.5 35B	54	52	45	0	0	30.4%
Z.AI GLM 4.5	50	48	46	0	0	28.9%
Llama 3.1 70B	72	71	0	0	0	28.7%
Gemini 3.5 Flash (Reasoning)	100	43	0	0	0	28.5%
Gemini 3.1 Flash Lite (Reasoning)	50	48	42	0	0	27.9%
Qwen 3 32B	100	39	0	0	0	27.8%
GPT-5	48	37	36	18	0	27.8%
Qwen 3.5 122B	78	44	17	0	0	27.7%
WizardLM 2 8x22b	41	39	31	27	0	27.6%
DeepSeek V4 Flash	100	36	0	0	0	27.2%
ByteDance Seed 1.6	100	30	0	0	0	26.0%
Gemini 2.5 Flash	81	45	0	0	0	25.3%
Qwen 3.6 Flash	51	28	27	18	0	24.9%
GPT-4o, Aug. 6th (temp=0)	43	41	40	0	0	24.6%
GPT-5 Nano	57	26	17	16	0	23.1%
o4 Mini High	73	23	17	0	0	22.6%
GPT-5.2	37	36	19	18	0	22.1%
Qwen 3.6 35B	76	19	16	0	0	22.0%
Gemini 3.1 Flash Lite (Preview)	56	49	0	0	0	21.1%
GPT-4o, Aug. 6th (temp=1)	52	51	0	0	0	20.5%
Qwen 3.6 27B	85	17	0	0	0	20.3%
ByteDance Seed 2.0 Mini	100	0	0	0	0	20.0%
Claude Opus 4	53	39	0	0	0	18.3%
Qwen3.6 Max Preview	39	34	19	0	0	18.3%
Mistral Small 3.2 24B	44	36	6	0	0	17.2%
Gemma 4 26B	47	30	0	0	0	15.5%
Grok 4.3	36	23	0	0	0	11.8%
Qwen3.7 Max	26	21	0	0	0	9.5%
ByteDance Seed 2.0 Lite	42	0	0	0	0	8.4%
Gemini 3 Flash (Preview, Reasoning)	25	16	0	0	0	8.2%
Qwen 3.5 27B	34	0	0	0	0	6.7%
Qwen 2.5 72B	30	0	0	0	0	6.0%
Qwen 3.5 Plus (2026-04-20)	29	0	0	0	0	5.7%
Gemini 3.1 Pro (Preview)	0	0	0	0	0	0.0%
Gemma 4 31B (Reasoning)	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Hermes 3 405B	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%

Adverb-first sentence starts

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets