AI-ism word frequency

Test: Bad Writing Habits

Avg. Score

42.2%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	ByteDance Seed 1.6 Flash	69.0%	$0.0013	27.3s	51%
2	Claude Sonnet 5	74.6%	$0.027	33.5s	55%
3	GPT-5.4 Mini (Reasoning)	71.2%	$0.022	28.1s	55%
4	GPT-5.4 Mini	68.1%	$0.015	16.8s	53%
5	ByteDance Seed 2.0 Lite	76.3%	$0.012	2.2m	63%
6	GPT-5.4 Mini (Reasoning, Low)	68.0%	$0.015	16.8s	51%
7	Claude Sonnet 5 (Reasoning)	73.6%	$0.030	38.9s	54%
8	Claude Opus 4.7	79.2%	$0.069	30.4s	60%
9	Claude Opus 4.7 (Reasoning)	78.1%	$0.076	32.0s	58%
10	Claude Sonnet 5 (Reasoning, Low)	70.0%	$0.031	38.4s	49%
11	GPT-5 Mini	66.0%	$0.0100	57.4s	49%
12	Claude Sonnet 4.6	70.0%	$0.031	39.3s	48%
13	GPT-5.4	73.9%	$0.049	1.4m	57%
14	GPT-5.4 (Reasoning, Low)	73.0%	$0.055	1.4m	58%
15	GPT-5.4 Nano (Reasoning, Low)	57.9%	$0.0055	20.6s	42%
16	Claude Opus 4.8 (Reasoning)	71.6%	$0.071	41.7s	53%
17	GPT-5.4 Nano	55.5%	$0.0057	26.3s	42%
18	Z.AI GLM 5.2 (Reasoning, High)	61.5%	$0.011	1.0m	44%
19	GPT-5.4 Nano (Reasoning)	56.3%	$0.0061	24.5s	40%
20	MiniMax M3	71.2%	$0.0060	3.1m	54%
21	MiniMax M2.5	60.5%	$0.0034	1.3m	44%
22	Claude Opus 4.8 (Reasoning, Low)	71.5%	$0.071	41.9s	49%
23	Z.AI GLM 5 Turbo	60.3%	$0.0081	33.2s	37%
24	Claude Sonnet 4.6 (Reasoning)	71.3%	$0.060	1.2m	50%
25	Claude Haiku 4.5	57.4%	$0.011	21.6s	38%
26	Grok 4.5 (Reasoning, Low)	60.4%	$0.018	1.1m	43%
27	MiniMax M2.7	58.4%	$0.0040	1.1m	40%
28	DeepSeek V4 Flash	54.9%	$0.0006	31.6s	35%
29	GPT-5	74.2%	$0.065	2.8m	61%
30	DeepSeek V4 Pro	57.6%	$0.0048	1.3m	40%
31	DeepSeek V4 Flash (Reasoning)	52.8%	$0.0007	31.1s	33%
32	Z.AI GLM 5	54.8%	$0.0084	1.2m	39%
33	Aion 3.0	58.4%	$0.024	1.0m	37%
34	Grok 4.20	50.1%	$0.0093	45.7s	37%
35	Grok 4.5 (Reasoning, High)	59.3%	$0.030	1.6m	43%
36	GPT-5.2	61.5%	$0.056	1.5m	48%
37	GPT-5.1	63.4%	$0.054	1.8m	48%
38	GPT-5.4 (Reasoning)	73.5%	$0.089	2.6m	58%
39	Grok 4.3	48.7%	$0.0069	30.5s	33%
40	Aion 3.0 Mini	56.1%	$0.0053	1.2m	32%
41	Qwen 3.5 Flash	47.5%	$0.0025	47.5s	34%
42	GPT-5.5	74.4%	$0.139	1.7m	62%
43	Claude Opus 4.6	63.6%	$0.078	1.2m	47%
44	Qwen 3.5 9B	55.4%	$0.0011	1.4m	32%
45	Claude Sonnet 4.5	54.6%	$0.035	38.1s	35%
46	Writer: Palmyra X5	46.6%	$0.011	22.0s	32%
47	GPT-5.5 (Reasoning)	74.8%	$0.142	1.8m	62%
48	GPT-5.5 (Reasoning, Low)	74.4%	$0.139	1.8m	60%
49	Gemini 3.5 Flash (Reasoning, Minimal)	44.5%	$0.018	12.0s	31%
50	GPT-5 Nano	46.6%	$0.0042	1.4m	36%
51	Z.AI GLM 4.7 Flash	45.8%	$0.0017	1.2m	33%
52	Gemini 3 Flash (Preview)	41.1%	$0.0078	19.6s	30%
53	Claude Opus 4.5	58.0%	$0.070	53.4s	41%
54	Z.AI GLM 5.1	52.7%	$0.014	1.5m	34%
55	ByteDance Seed 2.0 Mini	68.6%	$0.0045	4.9m	51%
56	Grok 4.20 (Reasoning)	49.4%	$0.018	1.5m	36%
57	Z.AI GLM 4.7	47.8%	$0.010	1.4m	34%
58	Claude Opus 4.6 (Reasoning)	62.7%	$0.088	1.4m	44%
59	Z.AI GLM 4.5	43.0%	$0.0051	42.1s	28%
60	Mistral Small 4 (Reasoning)	41.5%	$0.0022	30.2s	27%
61	Qwen3 235B A22B Instruct 2507	43.8%	$0.0011	59.2s	28%
62	Xiaomi MIMO v2.5	44.0%	$0.0054	31.8s	25%
63	Xiaomi MIMO v2.5 Pro	45.5%	$0.0085	53.5s	26%
64	Mistral Medium 3.1	39.0%	$0.0048	36.5s	27%
65	Mistral Small 4	37.0%	$0.0014	18.2s	25%
66	ByteDance Seed 1.6	54.6%	$0.013	2.5m	34%
67	Qwen 3.5 122B	46.4%	$0.025	1.1m	31%
68	Qwen 3.5 Plus (2026-02-15)	38.7%	$0.0060	31.5s	26%
69	Qwen 3.5 35B	47.0%	$0.018	1.0m	27%
70	Qwen 3 32B	40.4%	$0.0015	54.6s	25%
71	Cydonia 24B V4.1	38.8%	$0.0014	44.8s	25%
72	Gemini 3 Flash (Preview, Reasoning)	37.3%	$0.012	30.1s	26%
73	DeepSeek V4 Pro (Reasoning)	54.2%	$0.015	3.1m	37%
74	Aion 2.0	40.3%	$0.0064	1.3m	28%
75	Grok 4.3 (Reasoning)	50.3%	$0.021	2.3m	33%
76	Ministral 3 14B	33.3%	$0.0007	11.7s	20%
77	Z.AI GLM 4.5 Air	38.0%	$0.0029	58.2s	23%
78	GPT-4.1	38.0%	$0.018	44.7s	25%
79	DeepSeek V3 (2025-03-24)	35.9%	$0.0014	39.4s	20%
80	Mistral Large 2	36.8%	$0.013	29.4s	21%
81	Qwen 3.6 Flash	36.3%	$0.010	41.4s	22%
82	Mistral Large 3	34.9%	$0.0033	30.3s	20%
83	Qwen 3.5 27B	44.5%	$0.020	1.6m	28%
84	DeepSeek V3.2	39.2%	$0.0014	1.9m	26%
85	Claude Sonnet 4	39.2%	$0.032	43.7s	24%
86	Ministral 8B	27.1%	$0.0004	10.4s	15%
87	Qwen 3.5 397B A17B	46.8%	$0.014	3.0m	31%
88	Qwen 3.6 35B	33.5%	$0.0083	1.0m	21%
89	Gemini 3.5 Flash (Reasoning)	42.3%	$0.071	37.6s	29%
90	Z.AI GLM 4.6	31.3%	$0.0065	51.5s	18%
91	Ministral 3 8B	28.3%	$0.0008	19.6s	14%
92	Gemini 3.1 Flash Lite (Preview)	23.8%	$0.0030	8.4s	15%
93	MoonshotAI: Kimi K2.5	46.4%	$0.019	3.2m	31%
94	Qwen 3.5 Plus (2026-04-20)	35.5%	$0.017	1.8m	24%
95	Hermes 3 70B	29.9%	$0.0010	1.2m	18%
96	Hermes 3 405B	31.9%	$0.0032	53.2s	14%
97	Gemini 3.1 Flash Lite	23.3%	$0.0030	12.1s	13%
98	Gemini 3.1 Flash Lite (Reasoning)	22.5%	$0.0030	11.9s	13%
99	WizardLM 2 8x22b	33.8%	$0.0026	1.8m	17%
100	DeepSeek V3.1	30.3%	$0.0020	1.8m	19%
101	Gemma 3 27B	21.0%	$0.0006	52.6s	16%
102	DeepSeek V3 (2024-12-26)	23.8%	$0.0021	54.6s	13%
103	Gemini 2.5 Pro	26.5%	$0.036	36.2s	16%
104	DeepSeek-V2 Chat	21.9%	$0.0021	53.3s	12%
105	Gemma 4 31B	24.8%	$0.0010	1.6m	16%
106	Ministral 3B	17.3%	$0.0001	8.1s	7%
107	Qwen 3.6 27B	33.6%	$0.025	2.3m	22%
108	Gemma 4 26B	19.6%	$0.0009	55.1s	12%
109	o4 Mini	17.6%	$0.015	25.7s	12%
110	Ministral 3 3B	13.5%	$0.0005	11.1s	5%
111	Qwen3.7 Max	38.8%	$0.068	2.3m	25%
112	Gemma 4 31B (Reasoning)	23.8%	$0.0014	2.2m	14%
113	Arcee AI: Trinity Mini	12.9%	$0.0003	9.2s	0%
114	Gemini 2.5 Flash	12.6%	$0.0052	10.6s	2%
115	Nemotron 3 Super	16.4%	$0.0000	1.4m	10%
116	Gemma 3 12B	13.0%	$0.0004	41.3s	5%
117	Mistral NeMO	10.6%	$0.0005	10.1s	1%
118	o4 Mini High	16.1%	$0.025	47.2s	9%
119	Qwen3.6 Max Preview	39.6%	$0.050	3.5m	25%
120	GPT-4.1 Mini	11.7%	$0.0027	19.0s	0%
121	Llama 3.1 70B	13.3%	$0.0015	29.4s	0%
122	Cohere Command R+ (Aug. 2024)	19.9%	$0.020	52.5s	4%
123	Gemini 2.5 Flash Lite	7.9%	$0.0009	9.5s	0%
124	GPT-4.1 Nano	8.0%	$0.0007	13.3s	0%
125	Claude Opus 4	54.6%	$0.209	1.4m	37%
126	Gemma 3 4B	8.2%	$0.0002	20.0s	0%
127	Gemini 2.5 Flash (Reasoning)	11.1%	$0.011	21.5s	0%
128	Gemma 4 26B (Reasoning)	15.3%	$0.0013	2.0m	10%
129	Inception Mercury 2	5.1%	$0.0032	7.0s	0%
130	Gemini 2.5 Flash Lite (Reasoning)	8.9%	$0.0028	30.8s	0%
131	Gemini 3.1 Pro (Preview)	34.5%	$0.107	1.8m	23%
132	Qwen 2.5 72B	7.9%	$0.0010	36.7s	0%
133	GPT-4o Mini (temp=1)	2.3%	$0.0012	34.8s	0%
134	MoonshotAI: Kimi K2.6	52.1%	$0.058	6.5m	35%
135	GPT-4o Mini (temp=0)	0.8%	$0.0012	34.8s	0%
136	Nemotron 3 Nano	6.2%	$0.0010	1.1m	0%
137	GPT-OSS 120B	9.9%	$0.0015	1.8m	4%
138	GPT-4o, Aug. 6th (temp=1)	2.8%	$0.018	24.4s	0%
139	GPT-4o, Aug. 6th (temp=0)	3.1%	$0.023	22.7s	0%
140	Mistral Small 3.2 24B	11.9%	$0.0069	5.7m	0%
42.16%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
ByteDance Seed 2.0 Lite	89	79	77	75	70	78.0%
Claude Opus 4.7	81	81	67	66	62	71.3%
GPT-5.5	78	77	74	69	58	71.1%
GPT-5.4	77	72	69	66	61	69.0%
Claude Opus 4.7 (Reasoning)	82	77	70	63	48	68.1%
ByteDance Seed 1.6 Flash	82	77	69	55	48	66.1%
Claude Opus 4.8 (Reasoning)	77	75	63	54	45	62.8%
GPT-5.4 Mini	71	66	65	58	53	62.7%
GPT-5	75	63	63	56	56	62.4%
GPT-5.4 Mini (Reasoning)	72	65	62	60	51	62.0%
GPT-5.4 (Reasoning)	71	64	59	58	49	60.3%
Claude Sonnet 4.6 (Reasoning)	82	78	62	46	32	60.1%
GPT-5.5 (Reasoning, Low)	74	68	65	49	38	58.9%
GPT-5.4 Mini (Reasoning, Low)	66	62	61	55	50	58.9%
GPT-5.4 (Reasoning, Low)	71	61	58	52	52	58.8%
GPT-5 Mini	76	60	59	51	48	58.7%
GPT-5.5 (Reasoning)	60	59	58	57	56	58.1%
Claude Sonnet 5 (Reasoning)	80	61	53	49	47	58.0%
GPT-5.2	67	66	59	51	44	57.4%
Grok 4.5 (Reasoning, Low)	63	61	52	51	50	55.5%
ByteDance Seed 2.0 Mini	78	60	59	42	35	54.8%
Claude Sonnet 5	71	68	53	45	35	54.1%
Claude Opus 4.8 (Reasoning, Low)	71	68	56	40	35	54.0%
Claude Sonnet 5 (Reasoning, Low)	62	59	56	53	32	52.3%
Claude Sonnet 4.6	65	65	50	50	28	51.7%
GPT-5.1	68	60	43	41	37	49.8%
Claude Opus 4.6	64	49	46	45	44	49.5%
MiniMax M3	56	52	52	43	37	48.2%
Qwen 3.5 35B	73	64	36	32	29	47.0%
Claude Opus 4.5	64	54	53	39	15	45.1%
Claude Opus 4	61	51	47	35	32	45.0%
Claude Opus 4.6 (Reasoning)	63	57	53	27	21	44.4%
Qwen 3.5 9B	66	54	50	49	2	44.1%
Z.AI GLM 5.2 (Reasoning, High)	53	47	45	40	33	43.7%
ByteDance Seed 1.6	63	50	37	36	30	43.0%
MiniMax M2.5	67	62	34	27	25	42.8%
Grok 4.20 (Reasoning)	62	55	41	34	19	41.9%
GPT-5.4 Nano (Reasoning)	54	43	42	38	31	41.6%
Qwen 3.5 Flash	57	51	41	39	16	41.0%
Z.AI GLM 4.7	53	46	40	33	30	40.2%
GPT-5.4 Nano (Reasoning, Low)	49	44	44	35	29	40.2%
Z.AI GLM 4.7 Flash	59	54	46	41	0	40.0%
Writer: Palmyra X5	63	58	54	21	0	39.2%
Z.AI GLM 4.5	54	54	48	20	19	39.1%
Qwen 3.5 397B A17B	56	51	43	25	20	38.9%
Grok 4.20	51	45	43	41	13	38.8%
Z.AI GLM 5.1	56	54	43	28	11	38.3%
Aion 3.0	55	54	42	35	0	37.2%
Qwen3 235B A22B Instruct 2507	50	43	30	29	28	36.1%
Qwen 3.5 Plus (2026-02-15)	69	46	32	24	10	36.0%
GPT-5 Nano	41	41	36	34	27	35.7%
Grok 4.5 (Reasoning, High)	57	46	41	27	6	35.5%
DeepSeek V4 Pro (Reasoning)	46	45	35	35	15	35.3%
Z.AI GLM 5 Turbo	57	50	31	30	8	35.3%
Gemini 3.5 Flash (Reasoning, Minimal)	56	51	46	11	10	34.8%
Grok 4.3 (Reasoning)	51	48	41	23	10	34.7%
Qwen 3.5 Plus (2026-04-20)	64	45	39	13	12	34.6%
Cydonia 24B V4.1	61	42	32	18	17	34.0%
Claude Haiku 4.5	45	35	32	32	24	33.6%
MiniMax M2.7	46	39	34	34	15	33.5%
DeepSeek V3 (2025-03-24)	54	43	37	32	0	33.3%
Qwen 3.5 27B	45	44	30	24	22	32.7%
GPT-5.4 Nano	43	36	32	32	21	32.6%
Z.AI GLM 4.5 Air	60	41	40	9	6	31.5%
Qwen 3.5 122B	48	43	27	20	15	30.6%
Grok 4.3	44	41	37	29	0	30.2%
Z.AI GLM 5	43	35	27	25	18	29.4%
Xiaomi MIMO v2.5 Pro	48	34	32	21	11	29.4%
Gemini 3 Flash (Preview)	45	39	28	20	15	29.2%
MoonshotAI: Kimi K2.5	62	24	23	21	13	28.7%
Hermes 3 70B	58	33	28	25	0	28.7%
Claude Sonnet 4.5	40	35	30	24	13	28.1%
DeepSeek V4 Pro	44	37	28	21	10	28.1%
Qwen 3 32B	48	46	37	7	0	27.7%
DeepSeek V4 Flash	46	34	29	22	7	27.6%
Mistral Small 4 (Reasoning)	52	37	27	14	2	26.4%
Gemini 3.5 Flash (Reasoning)	42	38	25	22	2	25.7%
Aion 3.0 Mini	36	36	28	18	9	25.3%
Qwen 3.6 Flash	50	28	26	22	0	25.1%
Aion 2.0	44	34	17	16	13	24.8%
Mistral Medium 3.1	35	32	26	18	12	24.7%
WizardLM 2 8x22b	53	38	31	0	0	24.4%
Gemini 3.1 Pro (Preview)	43	40	26	10	0	23.8%
Mistral Large 3	34	33	18	16	13	22.9%
Ministral 3 14B	49	43	21	1	0	22.7%
Qwen3.7 Max	33	31	30	17	0	22.1%
DeepSeek V3.2	47	24	21	18	0	22.0%
DeepSeek V4 Flash (Reasoning)	33	29	20	16	8	21.5%
Mistral Small 4	44	40	20	0	0	20.9%
Hermes 3 405B	39	28	24	14	0	20.9%
DeepSeek V3 (2024-12-26)	41	39	25	0	0	20.9%
Xiaomi MIMO v2.5	25	22	20	19	15	20.1%
MoonshotAI: Kimi K2.6	46	34	18	0	0	19.6%
GPT-4.1	30	20	19	14	13	19.5%
Mistral Large 2	41	22	19	15	0	19.5%
Qwen3.6 Max Preview	42	30	22	3	0	19.2%
Llama 3.1 70B	88	0	0	0	0	17.6%
Gemini 2.5 Pro	33	28	14	13	0	17.5%
Ministral 3B	67	20	0	0	0	17.4%
Gemini 3.1 Flash Lite	38	37	4	0	0	15.8%
Qwen 3.6 27B	31	21	18	5	0	14.9%
Ministral 3 8B	28	26	15	0	0	13.9%
Gemma 4 26B	32	10	10	1	0	10.5%
Z.AI GLM 4.6	19	14	12	5	0	10.1%
Cohere Command R+ (Aug. 2024)	50	0	0	0	0	10.1%
Gemini 3.1 Flash Lite (Preview)	23	20	8	0	0	10.0%
Gemini 3.1 Flash Lite (Reasoning)	21	14	8	4	3	9.9%
Claude Sonnet 4	37	5	3	0	0	9.0%
DeepSeek-V2 Chat	30	14	0	0	0	8.7%
Arcee AI: Trinity Mini	37	0	0	0	0	7.3%
DeepSeek V3.1	18	12	7	0	0	7.3%
Gemini 3 Flash (Preview, Reasoning)	13	12	9	2	0	7.2%
Ministral 8B	29	5	0	0	0	6.8%
Gemma 4 31B	14	8	6	2	0	5.9%
Mistral NeMO	11	10	0	0	0	4.3%
Gemma 3 27B	13	7	0	0	0	4.1%
Qwen 3.6 35B	10	8	0	0	0	3.8%
Nemotron 3 Super	15	0	0	0	0	3.1%
Ministral 3 3B	12	0	0	0	0	2.4%
Gemini 2.5 Flash	5	0	0	0	0	1.1%
Gemma 3 4B	3	0	0	0	0	0.7%
Mistral Small 3.2 24B	3	0	0	0	0	0.6%
Nemotron 3 Nano	2	0	0	0	0	0.4%
Gemma 4 31B (Reasoning)	0	0	0	0	0	0.0%
Gemma 4 26B (Reasoning)	0	0	0	0	0	0.0%
o4 Mini High	0	0	0	0	0	0.0%
o4 Mini	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
ByteDance Seed 2.0 Mini	87	84	80	71	61	76.5%
GPT-5.5 (Reasoning)	75	71	71	70	70	71.6%
GPT-5	74	73	70	69	65	70.1%
GPT-5.4	76	76	75	66	58	70.0%
GPT-5.5	75	73	68	68	59	68.9%
Claude Opus 4.7 (Reasoning)	77	71	70	67	55	68.1%
GPT-5.5 (Reasoning, Low)	77	71	64	62	60	66.5%
Claude Sonnet 5 (Reasoning, Low)	81	77	66	55	49	65.7%
Claude Sonnet 5	85	76	67	51	48	65.5%
GPT-5.4 (Reasoning, Low)	79	76	66	57	43	64.3%
ByteDance Seed 2.0 Lite	79	71	58	57	44	61.8%
GPT-5.4 (Reasoning)	71	70	58	55	53	61.4%
GPT-5.4 Mini	78	66	61	49	47	60.2%
ByteDance Seed 1.6 Flash	75	63	58	54	48	59.5%
MiniMax M3	75	60	60	52	49	59.3%
Claude Opus 4.8 (Reasoning)	82	74	58	45	37	59.2%
GPT-5.4 Mini (Reasoning, Low)	68	61	59	53	51	58.5%
Claude Opus 4.7	76	66	51	47	43	56.7%
Claude Sonnet 5 (Reasoning)	68	67	58	57	30	56.1%
Claude Opus 4.6	70	66	49	46	41	54.6%
Claude Opus 4.8 (Reasoning, Low)	63	57	55	50	43	53.6%
GPT-5.1	73	59	50	49	36	53.6%
GPT-5.4 Mini (Reasoning)	77	52	50	49	38	53.2%
Qwen 3.5 27B	67	54	52	40	40	50.8%
Grok 4.5 (Reasoning, Low)	63	56	52	42	36	50.0%
GPT-5.2	65	51	50	50	19	47.0%
GPT-5 Mini	62	60	49	35	25	46.4%
MiniMax M2.7	68	49	49	38	28	46.4%
Qwen 3.5 9B	82	49	43	43	14	46.3%
MiniMax M2.5	66	57	54	27	27	46.3%
GPT-5.4 Nano (Reasoning)	57	52	47	37	36	45.7%
Claude Opus 4	64	60	37	36	29	45.4%
DeepSeek V4 Pro	56	50	42	40	39	45.3%
Qwen 3.5 397B A17B	58	53	44	40	31	45.3%
Claude Sonnet 4.5	74	52	47	34	18	45.2%
Claude Opus 4.6 (Reasoning)	61	52	45	40	27	45.0%
Claude Haiku 4.5	69	49	41	35	31	44.9%
DeepSeek V4 Flash	69	56	43	29	27	44.9%
Grok 4.5 (Reasoning, High)	58	45	41	41	36	44.2%
Aion 3.0 Mini	88	52	32	25	23	43.8%
Grok 4.3	55	49	45	45	24	43.7%
Qwen 3 32B	62	51	45	35	23	43.2%
Grok 4.20	53	51	37	37	32	42.2%
Qwen 3.5 Flash	62	49	43	37	15	41.3%
Z.AI GLM 5 Turbo	73	59	51	12	10	41.2%
Qwen 3.5 122B	55	52	42	34	23	41.2%
ByteDance Seed 1.6	46	45	40	35	35	40.2%
GPT-5.4 Nano (Reasoning, Low)	66	54	30	27	25	40.2%
Z.AI GLM 4.7	55	52	44	35	13	40.0%
Grok 4.20 (Reasoning)	44	42	42	38	33	39.7%
Claude Opus 4.5	48	44	40	31	31	38.8%
GPT-5 Nano	49	40	38	35	30	38.4%
Z.AI GLM 4.7 Flash	51	51	40	32	18	38.3%
Claude Sonnet 4.6	60	37	36	32	26	38.1%
DeepSeek V4 Pro (Reasoning)	56	41	37	33	22	37.9%
Claude Sonnet 4.6 (Reasoning)	55	47	34	33	20	37.9%
Z.AI GLM 5.2 (Reasoning, High)	68	63	23	20	14	37.6%
Z.AI GLM 5	63	40	38	25	21	37.3%
Z.AI GLM 5.1	64	35	33	30	23	36.8%
MoonshotAI: Kimi K2.5	61	38	31	29	23	36.2%
Qwen 3.5 35B	44	42	40	32	23	36.2%
Gemini 3.5 Flash (Reasoning)	44	36	34	33	31	35.6%
MoonshotAI: Kimi K2.6	43	39	36	33	27	35.5%
Gemini 3 Flash (Preview, Reasoning)	40	38	36	35	25	34.8%
GPT-5.4 Nano	46	44	32	25	22	33.7%
Qwen 3.5 Plus (2026-02-15)	52	37	32	31	15	33.6%
Hermes 3 405B	60	54	39	13	0	33.2%
DeepSeek V4 Flash (Reasoning)	64	52	24	22	4	33.0%
DeepSeek V3 (2025-03-24)	66	53	39	7	0	32.9%
Gemini 3 Flash (Preview)	51	38	29	28	20	32.9%
Qwen3 235B A22B Instruct 2507	51	39	30	28	11	31.8%
Qwen3.7 Max	41	31	31	27	24	30.9%
Grok 4.3 (Reasoning)	60	33	31	29	0	30.6%
Qwen 3.6 Flash	68	46	23	12	0	29.9%
Xiaomi MIMO v2.5 Pro	43	37	25	23	15	28.4%
Qwen3.6 Max Preview	41	35	34	22	9	28.2%
WizardLM 2 8x22b	45	41	27	15	0	25.8%
Aion 3.0	42	39	39	7	2	25.7%
Writer: Palmyra X5	38	30	25	19	14	25.2%
Aion 2.0	34	34	29	20	9	25.1%
DeepSeek V3 (2024-12-26)	67	52	0	0	0	23.8%
Gemini 3.1 Flash Lite (Reasoning)	68	19	13	12	4	23.3%
Gemini 3.5 Flash (Reasoning, Minimal)	34	32	23	20	4	22.9%
DeepSeek V3.2	36	34	27	16	0	22.5%
Mistral Medium 3.1	32	30	23	16	9	22.1%
GPT-4.1	33	26	19	18	14	22.0%
Claude Sonnet 4	46	37	26	0	0	21.9%
Gemini 3.1 Pro (Preview)	46	40	19	3	0	21.7%
Cydonia 24B V4.1	32	27	20	18	0	19.4%
Gemini 3.1 Flash Lite	29	27	17	6	5	16.8%
Hermes 3 70B	61	19	0	0	0	16.2%
Z.AI GLM 4.5 Air	39	26	14	0	0	15.8%
Mistral Large 2	49	19	9	0	0	15.4%
DeepSeek-V2 Chat	75	0	0	0	0	15.0%
Qwen 3.5 Plus (2026-04-20)	29	28	10	7	0	14.6%
Xiaomi MIMO v2.5	39	15	12	4	1	14.3%
Mistral Small 4	28	22	18	0	0	13.5%
Z.AI GLM 4.5	34	16	15	1	0	13.3%
Llama 3.1 70B	62	0	0	0	0	12.4%
Qwen 3.6 27B	32	19	10	0	0	12.3%
Gemma 4 31B	23	15	12	10	0	12.0%
Mistral Small 4 (Reasoning)	33	18	6	4	0	11.9%
Ministral 3 8B	27	19	9	0	0	10.8%
Gemini 2.5 Pro	27	26	0	0	0	10.6%
Qwen 3.6 35B	29	18	6	0	0	10.5%
Gemini 3.1 Flash Lite (Preview)	24	15	8	0	0	9.3%
Gemma 3 27B	28	12	0	0	0	8.1%
Gemma 3 12B	25	0	0	0	0	5.1%
Mistral Small 3.2 24B	17	9	0	0	0	5.1%
Ministral 3 3B	18	6	0	0	0	4.7%
DeepSeek V3.1	17	6	0	0	0	4.6%
Ministral 3 14B	13	10	0	0	0	4.5%
Z.AI GLM 4.6	22	0	0	0	0	4.5%
GPT-OSS 120B	16	0	0	0	0	3.1%
Gemma 4 26B	6	2	0	0	0	1.7%
Nemotron 3 Nano	8	0	0	0	0	1.5%
Mistral Large 3	5	1	0	0	0	1.2%
Cohere Command R+ (Aug. 2024)	5	0	0	0	0	1.0%
Gemini 2.5 Flash	5	0	0	0	0	1.0%
Gemma 4 31B (Reasoning)	2	2	0	0	0	0.8%
Gemma 3 4B	2	0	0	0	0	0.4%
Gemini 2.5 Flash (Reasoning)	1	0	0	0	0	0.1%
Gemma 4 26B (Reasoning)	0	0	0	0	0	0.0%
o4 Mini High	0	0	0	0	0	0.0%
o4 Mini	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
Nemotron 3 Super	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
Ministral 8B	0	0	0	0	0	0.0%
Ministral 3B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7	100	96	95	91	85	93.4%
Claude Opus 4.7 (Reasoning)	92	92	91	85	81	88.3%
GPT-5	92	88	87	84	79	86.2%
Claude Sonnet 5 (Reasoning, Low)	100	86	86	79	72	84.4%
ByteDance Seed 2.0 Lite	92	90	89	85	65	84.2%
ByteDance Seed 1.6 Flash	94	91	89	76	72	84.2%
GPT-5.4	91	86	85	81	74	83.6%
Claude Opus 4.8 (Reasoning, Low)	96	95	81	75	68	83.1%
GPT-5.5 (Reasoning, Low)	93	83	82	81	74	82.4%
Claude Opus 4.8 (Reasoning)	91	86	84	77	72	81.8%
Claude Sonnet 4.6 (Reasoning)	97	91	79	72	67	81.1%
GPT-5.5 (Reasoning)	83	82	82	81	78	81.0%
MiniMax M3	93	89	78	74	69	80.7%
GPT-5.4 (Reasoning)	83	82	82	80	74	80.4%
GPT-5.4 (Reasoning, Low)	88	84	78	77	69	79.3%
GPT-5.4 Mini (Reasoning)	82	82	79	77	72	78.3%
GPT-5.5	81	81	80	77	73	78.2%
Qwen 3.5 27B	83	79	78	77	70	77.4%
GPT-5.1	82	82	82	72	69	77.3%
Claude Haiku 4.5	87	80	77	74	66	76.9%
GPT-5.4 Mini (Reasoning, Low)	86	85	77	71	65	76.8%
Claude Sonnet 5	91	79	78	69	66	76.6%
GPT-5.4 Mini	89	83	77	69	63	76.3%
Grok 4.3 (Reasoning)	84	82	82	69	58	74.9%
MiniMax M2.7	93	79	78	63	61	74.9%
GPT-5.2	82	78	78	69	66	74.5%
Claude Opus 4.6 (Reasoning)	81	80	75	73	63	74.1%
GPT-5 Mini	88	76	71	68	60	72.9%
Claude Sonnet 4.6	85	77	73	71	57	72.4%
Claude Opus 4.5	85	77	67	66	62	71.6%
Claude Sonnet 5 (Reasoning)	81	81	70	69	55	71.3%
Claude Sonnet 4.5	89	79	69	59	56	70.5%
MoonshotAI: Kimi K2.6	79	72	71	64	63	69.8%
Z.AI GLM 5 Turbo	78	75	75	64	57	69.7%
Claude Opus 4.6	78	77	72	61	61	69.7%
Aion 3.0 Mini	90	81	65	58	53	69.3%
Hermes 3 405B	93	77	68	54	52	68.7%
Xiaomi MIMO v2.5 Pro	76	72	69	67	59	68.6%
Claude Opus 4	81	79	68	63	51	68.5%
Z.AI GLM 4.7	81	73	73	64	51	68.4%
Z.AI GLM 5.2 (Reasoning, High)	78	76	65	64	58	68.3%
Grok 4.5 (Reasoning, High)	78	72	71	61	56	67.6%
ByteDance Seed 2.0 Mini	74	69	65	65	64	67.4%
GPT-5.4 Nano (Reasoning, Low)	78	73	69	58	58	67.3%
GPT-5.4 Nano	71	68	67	64	62	66.5%
Qwen 3.5 35B	84	80	71	65	32	66.4%
Qwen 3.5 397B A17B	80	71	69	58	53	66.4%
Z.AI GLM 5	72	70	67	63	58	66.1%
GPT-5.4 Nano (Reasoning)	73	67	63	62	60	64.9%
Qwen 3.5 122B	84	80	72	44	44	64.8%
DeepSeek V4 Pro	80	68	67	56	54	64.7%
Grok 4.5 (Reasoning, Low)	79	76	65	56	47	64.6%
Aion 3.0	78	68	67	62	47	64.4%
DeepSeek V4 Flash (Reasoning)	78	77	66	58	40	63.9%
Grok 4.3	72	65	62	60	57	63.3%
Z.AI GLM 5.1	71	68	61	58	57	63.0%
MiniMax M2.5	80	69	69	67	28	62.6%
Qwen 3.5 Flash	75	63	62	58	54	62.4%
Z.AI GLM 4.7 Flash	79	70	65	54	40	61.6%
Grok 4.20 (Reasoning)	72	67	61	58	45	60.9%
DeepSeek V4 Pro (Reasoning)	77	74	59	56	38	60.7%
Writer: Palmyra X5	74	72	61	58	39	60.7%
ByteDance Seed 1.6	77	69	66	47	44	60.7%
Qwen 3.5 9B	76	73	59	54	40	60.4%
Z.AI GLM 4.5 Air	97	64	52	42	35	57.7%
DeepSeek V3.2	67	63	62	56	40	57.6%
MoonshotAI: Kimi K2.5	68	60	58	56	45	57.2%
Gemma 4 31B (Reasoning)	67	67	52	52	45	56.5%
Gemini 3 Flash (Preview)	67	63	57	57	36	56.0%
Qwen3 235B A22B Instruct 2507	74	61	53	46	44	55.7%
Z.AI GLM 4.5	70	55	55	51	46	55.6%
Qwen3.7 Max	69	68	60	46	33	55.3%
Aion 2.0	73	63	62	38	38	54.8%
Gemini 3.5 Flash (Reasoning, Minimal)	66	61	59	50	36	54.4%
Gemini 3.5 Flash (Reasoning)	79	57	50	47	38	54.2%
Mistral Small 4 (Reasoning)	67	62	61	42	38	54.0%
DeepSeek V4 Flash	85	84	62	40	0	54.0%
Qwen 3.5 Plus (2026-02-15)	69	57	54	51	37	53.7%
Mistral Small 4	61	58	56	50	43	53.5%
GPT-4.1	79	65	53	39	30	53.2%
Claude Sonnet 4	67	65	53	51	31	53.2%
WizardLM 2 8x22b	66	57	53	47	42	53.1%
DeepSeek V3 (2025-03-24)	71	66	55	54	19	53.0%
Mistral Large 3	66	58	58	41	38	52.0%
Mistral Large 2	69	68	53	43	20	50.7%
Xiaomi MIMO v2.5	81	53	49	48	21	50.4%
Qwen 3.6 35B	60	58	56	40	36	50.0%
Gemini 3 Flash (Preview, Reasoning)	54	50	50	48	45	49.7%
Cydonia 24B V4.1	57	57	50	47	35	49.4%
Ministral 8B	77	51	50	36	33	49.3%
Qwen3.6 Max Preview	69	61	49	34	30	48.7%
Qwen 3.6 27B	67	48	45	41	40	48.3%
Qwen 3.5 Plus (2026-04-20)	62	47	45	45	43	48.2%
Qwen 3 32B	71	63	63	22	20	47.5%
Gemini 3.1 Pro (Preview)	60	53	44	41	35	46.6%
Grok 4.20	58	53	49	46	26	46.4%
GPT-5 Nano	50	48	45	45	44	46.3%
Mistral Medium 3.1	69	59	43	31	30	46.1%
Ministral 3 8B	59	58	54	54	0	45.1%
DeepSeek V3.1	56	52	48	35	27	43.7%
Gemini 3.1 Flash Lite	54	48	44	36	32	42.7%
Qwen 3.6 Flash	71	46	40	34	22	42.7%
Gemini 2.5 Pro	64	48	45	32	17	41.3%
DeepSeek V3 (2024-12-26)	78	60	36	20	10	40.9%
Hermes 3 70B	85	44	37	36	0	40.5%
Gemma 3 27B	45	39	38	37	36	39.0%
Gemma 4 31B	54	45	37	32	26	38.9%
Gemma 4 26B	57	46	40	26	23	38.4%
Ministral 3 14B	59	38	34	34	25	37.9%
Z.AI GLM 4.6	72	30	30	30	18	35.8%
o4 Mini	45	34	33	33	22	33.4%
Gemini 3.1 Flash Lite (Reasoning)	66	46	26	17	11	33.2%
Gemini 3.1 Flash Lite (Preview)	53	39	29	26	10	31.0%
Nemotron 3 Super	48	45	21	21	0	27.3%
Arcee AI: Trinity Mini	59	45	20	0	0	24.8%
DeepSeek-V2 Chat	34	32	25	20	0	22.4%
Gemma 4 26B (Reasoning)	44	36	22	5	4	22.3%
Cohere Command R+ (Aug. 2024)	59	44	0	0	0	20.7%
Gemma 3 12B	37	22	21	18	3	20.3%
Gemini 2.5 Flash (Reasoning)	47	38	9	0	0	18.9%
Llama 3.1 70B	71	20	0	0	0	18.3%
GPT-4.1 Mini	52	21	16	1	0	18.1%
Gemini 2.5 Flash Lite	29	26	18	12	0	16.9%
Ministral 3B	52	31	0	0	0	16.6%
Gemini 2.5 Flash	44	32	5	0	0	16.4%
Mistral NeMO	55	11	9	3	0	15.7%
GPT-OSS 120B	31	28	7	5	1	14.4%
Nemotron 3 Nano	32	32	4	0	0	13.5%
Gemini 2.5 Flash Lite (Reasoning)	39	15	12	0	0	13.2%
Mistral Small 3.2 24B	62	0	0	0	0	12.5%
o4 Mini High	38	10	8	0	0	11.1%
Ministral 3 3B	38	9	0	0	0	9.5%
Gemma 3 4B	38	0	0	0	0	7.6%
Qwen 2.5 72B	14	0	0	0	0	2.9%
Inception Mercury 2	13	0	0	0	0	2.7%
GPT-4o, Aug. 6th (temp=0)	11	0	0	0	0	2.2%
GPT-4o, Aug. 6th (temp=1)	7	4	0	0	0	2.2%
GPT-4.1 Nano	8	1	0	0	0	2.0%
GPT-4o Mini (temp=1)	7	2	0	0	0	1.8%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7 (Reasoning)	95	95	92	87	80	89.8%
Claude Opus 4.7	95	92	90	86	72	87.2%
GPT-5.4 Mini (Reasoning)	88	87	86	86	80	85.4%
Claude Sonnet 4.6	95	90	86	86	67	84.9%
GPT-5.4 (Reasoning)	97	88	79	79	76	83.8%
GPT-5.5 (Reasoning, Low)	87	83	81	79	79	81.8%
GPT-5.4	93	88	81	73	73	81.8%
GPT-5.4 (Reasoning, Low)	82	81	81	80	78	80.5%
GPT-5.5 (Reasoning)	85	84	81	77	76	80.4%
Claude Sonnet 5 (Reasoning)	91	85	79	76	71	80.3%
GPT-5.5	93	84	76	70	68	78.4%
ByteDance Seed 2.0 Lite	84	84	75	75	73	78.2%
Claude Opus 4.8 (Reasoning, Low)	86	81	77	75	72	78.2%
Claude Sonnet 5	100	79	72	68	68	77.3%
GPT-5	83	82	76	72	72	77.0%
Claude Sonnet 4.6 (Reasoning)	96	79	79	74	56	76.8%
GPT-5.4 Mini (Reasoning, Low)	83	83	75	75	68	76.8%
GPT-5.4 Mini	84	81	79	78	60	76.3%
DeepSeek V4 Flash (Reasoning)	84	82	72	71	68	75.6%
GPT-5.4 Nano (Reasoning)	85	76	76	69	66	74.5%
Z.AI GLM 5.2 (Reasoning, High)	76	74	74	74	74	74.2%
Claude Opus 4.6 (Reasoning)	83	77	76	68	66	74.1%
Claude Sonnet 5 (Reasoning, Low)	91	82	77	62	53	73.2%
ByteDance Seed 1.6 Flash	92	86	79	52	50	71.6%
Claude Haiku 4.5	80	77	73	69	55	70.8%
GPT-5.2	78	74	71	65	64	70.5%
Claude Opus 4.6	82	79	73	62	56	70.4%
GPT-5.4 Nano (Reasoning, Low)	76	72	71	68	65	70.2%
Claude Opus 4.8 (Reasoning)	86	82	65	63	54	70.0%
Qwen 3.5 9B	85	77	71	60	57	70.0%
GPT-5 Mini	72	71	70	70	64	69.4%
DeepSeek V4 Pro (Reasoning)	80	79	66	63	54	68.3%
GPT-5.1	75	74	67	63	62	68.2%
GPT-5.4 Nano	78	76	70	60	54	67.6%
Claude Opus 4	72	71	71	64	58	67.2%
DeepSeek V4 Pro	71	70	69	63	62	67.1%
MiniMax M2.5	87	76	68	53	52	67.1%
Z.AI GLM 5	71	70	67	65	59	66.4%
Claude Sonnet 4.5	78	71	70	65	44	65.4%
Grok 4.5 (Reasoning, Low)	81	68	66	61	52	65.3%
Qwen 3.5 35B	82	65	64	61	53	64.9%
Claude Opus 4.5	85	84	59	50	44	64.5%
Mistral Large 2	69	64	64	61	57	62.9%
Z.AI GLM 5.1	78	77	73	46	38	62.6%
Z.AI GLM 5 Turbo	81	81	71	69	7	61.8%
ByteDance Seed 1.6	76	71	67	47	42	60.8%
WizardLM 2 8x22b	73	65	56	55	50	59.8%
Grok 4.20	73	66	54	53	52	59.7%
ByteDance Seed 2.0 Mini	75	65	62	54	40	59.1%
MiniMax M2.7	85	66	65	44	35	59.1%
Hermes 3 405B	92	60	60	54	29	58.9%
Writer: Palmyra X5	72	63	63	54	42	58.8%
Claude Sonnet 4	72	70	58	47	47	58.4%
MiniMax M3	76	65	64	50	37	58.3%
DeepSeek V4 Flash	76	66	59	53	34	57.5%
Mistral Small 4 (Reasoning)	69	62	56	51	46	56.5%
Aion 3.0	74	73	47	46	42	56.4%
Cydonia 24B V4.1	76	67	57	53	25	55.6%
Qwen 3.5 Flash	66	58	53	52	44	54.5%
Z.AI GLM 4.5 Air	83	65	59	47	17	54.5%
Xiaomi MIMO v2.5 Pro	69	56	55	50	40	54.0%
Grok 4.5 (Reasoning, High)	65	57	51	49	48	53.9%
Z.AI GLM 4.7	64	57	54	47	47	53.7%
Qwen 3.5 122B	66	55	55	49	44	53.7%
Gemini 3.5 Flash (Reasoning, Minimal)	66	63	49	48	41	53.5%
Qwen3 235B A22B Instruct 2507	79	63	61	43	20	53.3%
Grok 4.20 (Reasoning)	69	60	53	50	34	53.2%
Xiaomi MIMO v2.5	69	65	54	45	30	52.5%
Aion 3.0 Mini	69	63	47	45	40	52.5%
Z.AI GLM 4.5	82	53	49	49	29	52.3%
MoonshotAI: Kimi K2.5	60	59	50	43	37	49.8%
Grok 4.3 (Reasoning)	70	59	49	39	28	48.9%
Cohere Command R+ (Aug. 2024)	62	58	54	40	29	48.6%
Z.AI GLM 4.6	75	50	49	40	29	48.4%
Gemini 3.5 Flash (Reasoning)	62	57	45	40	37	48.1%
Grok 4.3	65	64	50	31	30	48.0%
Qwen 3.5 Plus (2026-04-20)	68	56	51	43	20	47.7%
GPT-5 Nano	53	52	44	44	43	47.2%
Mistral Medium 3.1	57	51	50	44	32	47.1%
Z.AI GLM 4.7 Flash	55	51	48	43	39	46.9%
MoonshotAI: Kimi K2.6	62	55	44	38	34	46.6%
Mistral Large 3	64	64	49	29	26	46.4%
DeepSeek V3.1	56	51	47	44	33	46.2%
Gemini 3 Flash (Preview)	65	49	44	42	29	45.7%
Qwen 3.5 27B	78	49	35	30	28	43.7%
Gemma 3 27B	60	46	46	35	28	43.0%
Ministral 3 14B	83	65	27	27	12	42.6%
Qwen 3.5 397B A17B	60	45	42	37	27	42.4%
Gemini 3 Flash (Preview, Reasoning)	53	51	43	42	23	42.3%
Mistral Small 3.2 24B	97	77	36	0	0	41.9%
Mistral Small 4	60	39	38	37	26	40.2%
DeepSeek V3.2	67	50	41	23	18	39.8%
Qwen 3 32B	45	42	37	36	34	38.9%
Aion 2.0	54	53	41	24	20	38.6%
Qwen3.7 Max	54	51	40	30	14	37.7%
GPT-4.1	52	40	38	37	21	37.7%
Gemini 3.1 Pro (Preview)	56	38	36	26	22	35.8%
Qwen 3.5 Plus (2026-02-15)	46	39	33	29	26	34.5%
Gemini 2.5 Pro	44	38	36	35	17	34.2%
Gemini 3.1 Flash Lite (Preview)	46	39	39	34	10	33.5%
Qwen 3.6 Flash	46	42	37	21	21	33.2%
Gemma 3 12B	50	43	29	23	21	33.1%
Hermes 3 70B	89	40	34	0	0	32.4%
Gemini 2.5 Flash	45	30	30	29	28	32.3%
Qwen 3.6 35B	66	42	33	11	0	30.4%
Ministral 3B	79	37	18	11	0	29.1%
Gemini 3.1 Flash Lite	62	53	19	9	0	28.6%
Qwen 3.6 27B	43	42	38	14	0	27.3%
DeepSeek-V2 Chat	47	30	27	18	13	27.2%
Ministral 8B	42	41	30	16	4	26.6%
Gemma 4 31B (Reasoning)	38	33	30	30	0	26.1%
Ministral 3 8B	69	33	28	0	0	25.8%
Gemma 4 26B (Reasoning)	46	26	25	25	7	25.6%
Gemma 4 31B	45	32	26	22	2	25.4%
Qwen3.6 Max Preview	34	29	26	20	11	24.0%
o4 Mini High	36	25	23	17	17	23.5%
DeepSeek V3 (2024-12-26)	30	29	26	17	15	23.5%
o4 Mini	40	33	20	19	0	22.6%
DeepSeek V3 (2025-03-24)	49	28	14	13	0	20.9%
Gemini 3.1 Flash Lite (Reasoning)	41	24	23	9	0	19.6%
Ministral 3 3B	33	24	23	18	0	19.6%
Gemma 4 26B	38	27	24	5	4	19.3%
Nemotron 3 Super	55	21	17	4	0	19.3%
Arcee AI: Trinity Mini	47	26	23	0	0	19.3%
Gemini 2.5 Flash Lite	51	37	0	0	0	17.6%
Gemini 2.5 Flash (Reasoning)	36	23	14	8	1	16.3%
GPT-4.1 Mini	42	26	12	0	0	16.0%
Gemini 2.5 Flash Lite (Reasoning)	49	12	10	0	0	14.1%
GPT-4.1 Nano	37	17	15	0	0	14.0%
Mistral NeMO	33	19	17	0	0	13.8%
Gemma 3 4B	21	15	10	10	2	11.6%
Llama 3.1 70B	32	15	0	0	0	9.3%
Nemotron 3 Nano	17	16	7	5	0	9.1%
GPT-4o Mini (temp=0)	26	19	0	0	0	8.9%
Inception Mercury 2	35	2	2	0	0	7.8%
Qwen 2.5 72B	30	2	0	0	0	6.4%
GPT-OSS 120B	17	9	0	0	0	5.2%
GPT-4o, Aug. 6th (temp=1)	17	4	3	0	0	4.9%
GPT-4o Mini (temp=1)	15	0	0	0	0	3.0%
GPT-4o, Aug. 6th (temp=0)	10	2	0	0	0	2.5%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6 (Reasoning)	100	93	92	91	81	91.3%
Claude Opus 4.7	95	91	90	87	81	88.7%
Claude Opus 4.8 (Reasoning, Low)	100	91	90	69	67	83.5%
Aion 3.0	89	89	76	71	65	77.9%
GPT-5.4 Mini (Reasoning)	79	78	78	77	74	77.1%
MiniMax M3	82	81	79	73	71	77.0%
Claude Sonnet 5	88	81	74	71	70	76.8%
Claude Sonnet 4.6	88	80	76	65	65	74.8%
ByteDance Seed 2.0 Lite	80	79	76	69	69	74.6%
Claude Opus 4.7 (Reasoning)	85	80	74	73	59	74.3%
Z.AI GLM 5 Turbo	85	80	74	67	65	74.0%
ByteDance Seed 2.0 Mini	85	84	76	66	58	73.8%
GPT-5.5 (Reasoning, Low)	82	76	74	68	67	73.5%
Z.AI GLM 5.2 (Reasoning, High)	82	76	71	67	67	72.7%
GPT-5.4 (Reasoning, Low)	78	78	71	68	68	72.4%
Claude Opus 4.8 (Reasoning)	86	81	81	68	46	72.3%
Claude Opus 4.6	84	82	74	62	57	71.8%
ByteDance Seed 1.6 Flash	79	78	74	71	56	71.7%
Claude Haiku 4.5	86	83	79	64	45	71.4%
GPT-5.5 (Reasoning)	82	76	73	67	58	71.3%
Qwen 3.5 9B	88	83	73	68	43	71.2%
GPT-5.4 (Reasoning)	77	74	72	66	66	71.0%
GPT-5.4 Mini (Reasoning, Low)	82	71	70	68	64	71.0%
Xiaomi MIMO v2.5 Pro	78	72	72	68	62	70.4%
GPT-5.4	79	78	75	68	52	70.4%
GPT-5	81	73	68	64	63	69.7%
Hermes 3 405B	95	94	57	55	48	69.6%
DeepSeek V4 Pro	82	75	70	64	55	69.3%
GPT-5.5	76	71	70	67	62	69.0%
GPT-5.1	79	77	65	64	59	68.6%
ByteDance Seed 1.6	91	77	63	56	54	68.2%
GPT-5.4 Mini	81	72	66	65	55	67.9%
Claude Sonnet 5 (Reasoning, Low)	76	75	74	59	55	67.8%
Claude Opus 4.5	77	74	68	62	54	67.1%
Aion 3.0 Mini	86	81	63	60	45	67.0%
Claude Opus 4.6 (Reasoning)	78	75	74	66	43	67.0%
GPT-5 Mini	75	70	69	62	57	66.9%
Qwen 3.5 35B	85	82	78	54	36	66.9%
Grok 4.5 (Reasoning, High)	73	71	65	63	61	66.5%
MiniMax M2.7	70	68	67	66	62	66.5%
WizardLM 2 8x22b	79	74	69	59	49	66.0%
MiniMax M2.5	69	69	68	65	56	65.4%
Claude Sonnet 5 (Reasoning)	76	70	69	61	51	65.2%
Z.AI GLM 5	79	65	62	61	59	65.1%
Mistral Large 2	71	69	68	66	50	64.8%
Claude Opus 4	75	74	70	51	51	64.4%
Qwen 3.5 397B A17B	77	67	65	55	54	63.8%
Qwen 3.6 35B	73	68	66	62	47	63.2%
Qwen 3.5 122B	84	64	62	54	52	63.1%
Grok 4.5 (Reasoning, Low)	74	68	62	60	49	62.6%
DeepSeek V4 Pro (Reasoning)	93	65	58	49	48	62.3%
GPT-5.4 Nano	65	65	61	60	56	61.5%
DeepSeek V4 Flash	93	68	65	55	20	60.4%
Qwen 3 32B	66	64	61	60	51	60.3%
Writer: Palmyra X5	74	73	66	58	25	59.2%
Claude Sonnet 4.5	74	73	71	49	29	59.2%
GPT-5.2	66	64	57	55	51	58.6%
GPT-5.4 Nano (Reasoning, Low)	61	60	59	58	52	58.2%
DeepSeek V4 Flash (Reasoning)	85	64	58	43	37	57.6%
Qwen 3.5 Flash	76	65	61	53	30	57.1%
Mistral Small 4 (Reasoning)	73	67	57	47	41	56.9%
Grok 4.3 (Reasoning)	67	63	63	49	42	56.7%
Grok 4.20 (Reasoning)	68	66	55	51	44	56.7%
MoonshotAI: Kimi K2.6	58	58	56	55	54	56.0%
GPT-5.4 Nano (Reasoning)	68	68	54	47	43	55.8%
Mistral Small 4	73	65	62	42	37	55.7%
MoonshotAI: Kimi K2.5	76	65	53	49	34	55.5%
Z.AI GLM 5.1	71	63	59	51	32	55.3%
Z.AI GLM 4.7	59	57	57	53	42	53.8%
Gemini 3.5 Flash (Reasoning, Minimal)	67	61	56	47	37	53.4%
Z.AI GLM 4.7 Flash	70	62	58	47	30	53.3%
Grok 4.3	69	55	54	50	34	52.4%
Gemini 3 Flash (Preview)	61	60	50	50	41	52.3%
Qwen 3.5 27B	67	62	51	45	36	52.1%
Claude Sonnet 4	69	62	56	46	27	52.0%
Qwen 3.5 Plus (2026-02-15)	59	57	51	47	45	52.0%
Xiaomi MIMO v2.5	61	59	55	43	42	51.8%
Qwen3.7 Max	58	57	49	48	42	51.0%
Gemini 2.5 Pro	62	53	52	48	32	49.5%
Qwen 3.6 27B	64	63	44	39	35	49.1%
Aion 2.0	55	53	53	49	32	48.5%
Gemini 3.1 Pro (Preview)	65	50	49	42	35	48.1%
Z.AI GLM 4.5 Air	54	52	48	43	42	47.8%
Hermes 3 70B	67	47	41	40	40	47.1%
Grok 4.20	58	55	49	47	23	46.5%
Qwen 3.6 Flash	56	53	51	36	35	46.3%
Z.AI GLM 4.5	58	56	41	37	36	45.5%
DeepSeek V3.1	70	59	45	31	22	45.4%
Qwen3.6 Max Preview	72	53	39	32	31	45.2%
Gemini 3 Flash (Preview, Reasoning)	55	48	46	44	33	45.0%
Ministral 3 14B	74	45	40	33	31	44.7%
Qwen3 235B A22B Instruct 2507	63	48	42	34	31	43.6%
DeepSeek V3.2	57	55	40	38	27	43.6%
Ministral 3B	75	57	37	26	21	43.1%
GPT-5 Nano	70	46	38	37	25	43.1%
DeepSeek V3 (2025-03-24)	69	64	41	40	0	42.8%
Mistral Large 3	75	39	37	34	27	42.2%
Gemma 4 26B	55	48	39	33	31	41.5%
Cydonia 24B V4.1	62	57	54	33	0	41.4%
GPT-4.1	46	42	37	37	33	39.1%
Mistral Medium 3.1	45	41	37	32	28	36.6%
Qwen 3.5 Plus (2026-04-20)	53	45	41	29	13	36.4%
Gemini 3.5 Flash (Reasoning)	49	39	34	32	26	36.0%
Z.AI GLM 4.6	46	42	39	33	19	35.9%
Gemini 3.1 Flash Lite	57	56	33	29	0	34.9%
DeepSeek V3 (2024-12-26)	48	37	27	26	19	31.6%
Ministral 8B	60	48	32	9	7	30.9%
Gemini 3.1 Flash Lite (Preview)	57	36	32	17	11	30.6%
Nemotron 3 Super	39	38	28	20	19	28.8%
Gemini 3.1 Flash Lite (Reasoning)	56	43	18	16	9	28.2%
Gemma 4 31B (Reasoning)	43	36	34	14	13	28.1%
Arcee AI: Trinity Mini	76	30	25	8	0	27.8%
Mistral Small 3.2 24B	48	44	21	20	0	26.5%
Mistral NeMO	51	35	32	5	2	24.7%
Gemma 4 31B	39	29	28	18	7	24.3%
Ministral 3 3B	53	45	13	0	0	22.2%
Gemma 3 12B	61	45	2	0	0	21.8%
Gemma 3 27B	45	26	26	3	3	20.7%
Ministral 3 8B	50	28	21	2	0	20.2%
Cohere Command R+ (Aug. 2024)	73	22	5	0	0	19.9%
Gemini 2.5 Flash (Reasoning)	37	29	17	8	6	19.4%
Gemma 4 26B (Reasoning)	31	29	23	8	4	18.9%
DeepSeek-V2 Chat	66	10	8	7	0	18.3%
o4 Mini High	31	23	23	0	0	15.5%
GPT-4.1 Nano	43	21	8	1	0	14.5%
Gemini 2.5 Flash	36	24	8	0	0	13.6%
GPT-OSS 120B	24	21	16	5	0	13.1%
o4 Mini	38	24	1	0	0	12.4%
Gemini 2.5 Flash Lite (Reasoning)	39	10	4	0	0	10.6%
GPT-4.1 Mini	26	8	7	0	0	8.2%
GPT-4o, Aug. 6th (temp=0)	38	0	0	0	0	7.6%
Llama 3.1 70B	35	0	0	0	0	6.9%
Nemotron 3 Nano	11	10	2	0	0	4.5%
GPT-4o Mini (temp=1)	12	0	0	0	0	2.5%
Gemini 2.5 Flash Lite	11	0	0	0	0	2.3%
Qwen 2.5 72B	3	0	0	0	0	0.5%
GPT-4o, Aug. 6th (temp=1)	3	0	0	0	0	0.5%
Inception Mercury 2	0	0	0	0	0	0.1%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7 (Reasoning)	95	88	86	77	77	84.7%
Claude Opus 4.8 (Reasoning)	95	91	81	78	75	84.1%
Claude Sonnet 5	85	85	85	85	76	83.4%
GPT-5.5 (Reasoning)	87	85	84	81	73	82.1%
ByteDance Seed 2.0 Lite	89	86	80	79	72	81.3%
Claude Opus 4.7	90	82	81	81	72	81.3%
Claude Opus 4.8 (Reasoning, Low)	87	86	82	76	71	80.2%
GPT-5.4 Mini (Reasoning)	88	86	79	75	68	79.1%
GPT-5.5 (Reasoning, Low)	92	78	76	75	70	78.3%
GPT-5.4 (Reasoning)	87	80	78	73	72	78.1%
GPT-5	85	82	80	75	67	77.7%
GPT-5.4 Mini (Reasoning, Low)	87	79	76	75	70	77.3%
GPT-5.4 (Reasoning, Low)	90	81	77	70	66	77.0%
GPT-5.4	83	80	76	73	72	77.0%
GPT-5.5	84	81	77	74	66	76.7%
MiniMax M2.5	91	80	80	73	56	76.2%
GPT-5 Mini	84	75	74	72	71	75.4%
Claude Sonnet 4.6	91	77	69	67	66	74.0%
GPT-5.1	88	82	72	65	62	74.0%
Grok 4.5 (Reasoning, High)	86	75	74	66	66	73.4%
Claude Sonnet 5 (Reasoning, Low)	86	82	80	76	43	73.4%
GPT-5.4 Mini	78	76	74	70	69	73.2%
ByteDance Seed 1.6 Flash	79	73	71	71	68	72.6%
Grok 4.5 (Reasoning, Low)	85	79	74	70	55	72.6%
Claude Sonnet 4.6 (Reasoning)	95	82	65	64	55	72.3%
MiniMax M3	81	76	76	64	64	72.0%
Z.AI GLM 5	89	83	82	59	43	71.2%
GPT-5.2	75	71	70	70	69	70.9%
Claude Opus 4	88	73	67	60	57	69.3%
Claude Opus 4.5	79	79	72	63	50	68.7%
GPT-5.4 Nano	69	69	67	66	64	67.1%
DeepSeek V4 Pro	90	79	64	62	38	66.4%
Claude Sonnet 4.5	79	78	76	71	27	66.3%
DeepSeek V4 Flash (Reasoning)	76	72	65	61	58	66.1%
Claude Sonnet 5 (Reasoning)	81	78	65	62	42	65.7%
Aion 3.0	80	79	69	51	49	65.6%
Z.AI GLM 5.2 (Reasoning, High)	82	76	71	62	37	65.6%
GPT-5.4 Nano (Reasoning, Low)	71	65	64	64	63	65.3%
Claude Opus 4.6	80	66	62	60	57	65.0%
GPT-5.4 Nano (Reasoning)	72	69	66	63	52	64.6%
Claude Opus 4.6 (Reasoning)	75	68	68	64	48	64.5%
DeepSeek V4 Pro (Reasoning)	73	66	64	61	57	64.3%
Aion 3.0 Mini	83	68	57	57	48	62.6%
Z.AI GLM 5 Turbo	91	64	57	51	49	62.5%
Z.AI GLM 4.5 Air	76	68	67	57	43	62.3%
MoonshotAI: Kimi K2.6	77	69	64	50	48	61.8%
ByteDance Seed 1.6	93	67	52	51	46	61.7%
Qwen 3.5 397B A17B	77	67	59	55	48	61.3%
Grok 4.20	79	61	59	54	53	61.2%
MiniMax M2.7	74	64	63	57	46	60.8%
Qwen 3.5 27B	85	69	62	50	37	60.7%
Claude Haiku 4.5	87	59	55	53	47	60.2%
DeepSeek V4 Flash	70	70	62	55	41	59.7%
ByteDance Seed 2.0 Mini	70	67	55	52	49	58.5%
Grok 4.3	70	68	66	55	32	58.4%
Qwen 3.6 Flash	78	73	51	41	37	55.9%
Z.AI GLM 5.1	82	55	53	49	41	55.8%
Xiaomi MIMO v2.5 Pro	81	78	57	44	19	55.7%
Qwen 3.5 122B	78	66	58	53	19	54.8%
Gemini 3.5 Flash (Reasoning)	66	61	58	46	42	54.7%
Z.AI GLM 4.7 Flash	71	60	54	47	40	54.5%
Qwen3.6 Max Preview	67	64	58	48	34	54.3%
Z.AI GLM 4.7	62	58	56	51	44	54.3%
Grok 4.3 (Reasoning)	68	62	56	53	32	54.2%
Grok 4.20 (Reasoning)	63	62	58	53	34	54.0%
Qwen3.7 Max	71	60	54	45	40	53.9%
Qwen3 235B A22B Instruct 2507	64	54	52	49	46	53.0%
Qwen 3.5 Flash	71	54	53	50	29	51.5%
Cydonia 24B V4.1	64	63	57	37	36	51.3%
Qwen 3 32B	79	69	56	53	0	51.2%
Qwen 3.5 35B	71	63	44	41	34	50.4%
Writer: Palmyra X5	59	57	49	46	39	50.1%
GPT-4.1	62	60	43	43	35	48.7%
MoonshotAI: Kimi K2.5	72	58	52	38	23	48.6%
GPT-5 Nano	56	55	48	42	41	48.5%
Claude Sonnet 4	80	54	38	36	33	48.2%
Qwen 3.5 9B	63	63	59	45	7	47.5%
DeepSeek V3 (2025-03-24)	78	51	48	36	19	46.4%
Mistral Medium 3.1	54	49	44	43	40	46.1%
Gemini 3.5 Flash (Reasoning, Minimal)	63	49	48	47	23	46.1%
Z.AI GLM 4.5	61	50	48	46	22	45.6%
DeepSeek V3.2	57	46	45	42	32	44.5%
WizardLM 2 8x22b	54	47	46	46	29	44.3%
Mistral Small 4 (Reasoning)	65	55	40	33	26	43.9%
DeepSeek V3.1	54	54	47	39	25	43.7%
Z.AI GLM 4.6	75	43	38	31	31	43.7%
Qwen 3.5 Plus (2026-02-15)	56	49	46	39	28	43.4%
Aion 2.0	58	43	42	41	24	41.4%
Qwen 3.5 Plus (2026-04-20)	58	51	40	34	24	41.4%
Qwen 3.6 27B	70	48	44	29	12	40.7%
Gemini 3 Flash (Preview)	56	50	40	32	22	39.9%
Qwen 3.6 35B	60	49	39	24	20	38.2%
Ministral 8B	58	44	32	31	19	36.7%
Mistral Large 2	59	37	37	30	20	36.5%
Gemini 3 Flash (Preview, Reasoning)	44	40	33	32	29	35.7%
DeepSeek V3 (2024-12-26)	59	48	46	20	0	34.7%
Mistral Small 4	58	36	27	26	20	33.5%
Mistral Large 3	50	48	24	23	21	33.3%
Ministral 3 14B	44	43	36	36	2	32.3%
Gemini 3.1 Pro (Preview)	43	38	29	28	23	32.1%
Xiaomi MIMO v2.5	39	38	36	26	22	32.1%
Gemma 4 31B	41	37	32	28	17	30.8%
Hermes 3 405B	46	43	38	25	0	30.4%
Gemini 2.5 Pro	49	47	29	22	0	29.4%
DeepSeek-V2 Chat	59	34	31	19	1	28.7%
Gemma 3 27B	42	38	29	29	0	27.6%
o4 Mini High	39	32	31	17	12	26.4%
Gemini 3.1 Flash Lite (Preview)	39	39	37	10	0	25.0%
Gemini 3.1 Flash Lite	45	33	18	18	7	24.0%
Ministral 3 3B	44	27	23	17	0	22.1%
Cohere Command R+ (Aug. 2024)	42	39	5	2	2	17.9%
Gemma 4 26B	46	22	14	5	0	17.6%
Ministral 3 8B	61	26	1	0	0	17.5%
Gemini 3.1 Flash Lite (Reasoning)	37	24	21	1	0	16.5%
Gemma 3 12B	42	21	14	6	0	16.4%
Arcee AI: Trinity Mini	37	35	8	1	0	16.2%
Gemma 4 26B (Reasoning)	34	15	13	10	6	15.7%
Gemma 4 31B (Reasoning)	33	20	18	6	0	15.2%
Ministral 3B	47	21	5	1	0	15.0%
Hermes 3 70B	47	17	7	0	0	14.2%
Qwen 2.5 72B	28	20	15	0	0	12.7%
Gemma 3 4B	34	13	11	5	0	12.5%
o4 Mini	26	16	8	0	0	9.8%
Gemini 2.5 Flash (Reasoning)	20	19	10	0	0	9.8%
Llama 3.1 70B	25	13	6	0	0	9.0%
Gemini 2.5 Flash Lite	21	11	2	0	0	7.0%
GPT-OSS 120B	15	14	3	0	0	6.2%
Gemini 2.5 Flash	26	2	0	0	0	5.7%
Gemini 2.5 Flash Lite (Reasoning)	19	8	1	0	0	5.6%
Mistral Small 3.2 24B	22	5	0	0	0	5.5%
Nemotron 3 Super	13	9	0	0	0	4.3%
Mistral NeMO	13	8	0	0	0	4.3%
GPT-4o Mini (temp=1)	19	0	0	0	0	3.7%
Nemotron 3 Nano	14	1	0	0	0	3.2%
Inception Mercury 2	3	0	0	0	0	0.6%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7 (Reasoning)	88	87	73	69	68	76.9%
Claude Opus 4.7	76	75	71	71	67	71.8%
Claude Sonnet 5 (Reasoning)	81	80	80	74	39	70.9%
GPT-5.5 (Reasoning)	76	74	67	66	61	68.9%
MiniMax M3	79	77	66	63	57	68.6%
ByteDance Seed 2.0 Lite	75	72	71	62	56	67.3%
Aion 3.0	81	75	66	65	49	67.0%
GPT-5.4 (Reasoning, Low)	75	67	66	62	55	65.2%
GPT-5.5 (Reasoning, Low)	74	67	66	63	53	64.7%
GPT-5	67	65	65	63	58	63.7%
GPT-5.5	75	69	65	57	49	63.1%
Claude Opus 4.8 (Reasoning, Low)	68	68	61	59	58	63.0%
Claude Sonnet 5 (Reasoning, Low)	77	62	61	58	53	61.9%
Claude Sonnet 4.6 (Reasoning)	76	69	67	52	43	61.3%
GPT-5.4	70	66	65	55	48	60.8%
Claude Sonnet 5	74	61	58	58	49	59.9%
Claude Opus 4.8 (Reasoning)	83	61	52	52	50	59.6%
Claude Sonnet 4.6	70	61	56	51	50	57.5%
GPT-5.4 (Reasoning)	60	59	56	55	51	56.0%
ByteDance Seed 1.6 Flash	73	63	52	46	41	54.9%
Grok 4.5 (Reasoning, High)	72	69	54	37	36	53.6%
GPT-5 Mini	71	67	48	44	36	53.4%
Claude Haiku 4.5	78	73	50	48	18	53.3%
GPT-5.2	58	54	52	51	51	53.0%
GPT-5.4 Mini	63	61	58	53	30	53.0%
ByteDance Seed 2.0 Mini	59	57	56	46	44	52.4%
Z.AI GLM 5.2 (Reasoning, High)	63	62	56	45	34	52.2%
Claude Opus 4.6 (Reasoning)	61	59	51	50	41	52.2%
Xiaomi MIMO v2.5	65	63	62	38	30	51.4%
Grok 4.5 (Reasoning, Low)	58	55	54	47	41	51.1%
GPT-5.4 Mini (Reasoning, Low)	54	51	50	49	48	50.3%
GPT-5.4 Mini (Reasoning)	64	55	48	44	39	50.1%
MiniMax M2.5	59	55	47	44	42	49.5%
Z.AI GLM 4.5	58	56	47	46	39	49.2%
MoonshotAI: Kimi K2.6	57	57	50	50	29	48.5%
Grok 4.20	60	51	43	41	38	46.5%
GPT-5.1	59	51	47	39	36	46.4%
Claude Sonnet 4.5	56	53	48	43	30	46.2%
Claude Opus 4.5	58	46	45	44	37	46.0%
Claude Opus 4.6	61	56	42	39	27	45.0%
DeepSeek V4 Flash	53	47	47	40	37	44.7%
Z.AI GLM 4.5 Air	59	55	52	42	15	44.5%
DeepSeek V4 Pro (Reasoning)	60	59	42	38	22	44.3%
DeepSeek V4 Pro	52	48	47	42	28	43.3%
Z.AI GLM 4.7 Flash	50	46	44	38	26	40.8%
Z.AI GLM 5 Turbo	56	49	35	30	30	40.2%
Aion 3.0 Mini	100	45	29	26	0	40.0%
Grok 4.20 (Reasoning)	44	42	37	37	34	38.7%
GPT-5 Nano	61	40	39	27	25	38.5%
Hermes 3 405B	55	53	50	26	9	38.4%
MiniMax M2.7	52	49	46	28	15	38.1%
Qwen 3.5 9B	46	43	37	32	32	38.0%
Qwen 3.5 Plus (2026-02-15)	53	38	38	36	23	37.7%
Qwen3 235B A22B Instruct 2507	54	39	35	33	28	37.7%
GPT-5.4 Nano	52	41	37	36	19	37.2%
Xiaomi MIMO v2.5 Pro	59	48	40	38	0	37.1%
Grok 4.3	50	50	48	24	14	37.1%
Claude Opus 4	48	39	39	28	25	35.8%
GPT-5.4 Nano (Reasoning, Low)	47	44	37	29	23	35.7%
ByteDance Seed 1.6	54	48	27	25	21	35.0%
Grok 4.3 (Reasoning)	45	40	37	30	22	34.9%
Z.AI GLM 5	53	45	39	27	10	34.6%
MoonshotAI: Kimi K2.5	49	47	30	23	22	34.3%
DeepSeek V4 Flash (Reasoning)	69	40	36	26	0	34.1%
Z.AI GLM 5.1	57	42	34	19	18	33.9%
Mistral Small 4 (Reasoning)	51	38	34	25	21	33.7%
Gemini 3.5 Flash (Reasoning)	47	38	35	28	14	32.4%
Z.AI GLM 4.7	42	31	30	29	28	31.9%
Mistral Medium 3.1	45	39	30	21	19	30.7%
Gemini 3.1 Pro (Preview)	41	35	34	31	11	30.3%
Writer: Palmyra X5	37	33	32	32	12	29.3%
Aion 2.0	47	27	26	24	20	28.7%
GPT-5.4 Nano (Reasoning)	32	32	31	23	22	28.2%
Qwen 3.5 35B	50	39	36	12	0	27.3%
Qwen 3.5 27B	41	39	30	15	8	26.6%
Mistral Small 4	39	34	27	20	12	26.4%
GPT-4.1	43	34	20	18	17	26.3%
Mistral Large 2	31	31	24	23	21	26.0%
Gemini 3 Flash (Preview, Reasoning)	48	37	23	16	0	24.8%
Ministral 3 8B	56	37	25	1	0	23.8%
Cydonia 24B V4.1	43	34	26	16	0	23.8%
Qwen 3 32B	49	27	24	17	0	23.3%
Mistral Large 3	34	29	23	15	11	22.5%
Gemini 3.5 Flash (Reasoning, Minimal)	50	39	14	7	0	22.1%
Qwen3.7 Max	47	28	20	12	0	21.4%
Claude Sonnet 4	37	22	16	15	15	20.9%
Qwen3.6 Max Preview	29	27	23	23	0	20.3%
Gemini 3.1 Flash Lite	37	29	15	11	0	18.4%
Gemini 3.1 Flash Lite (Preview)	40	32	12	5	0	17.9%
DeepSeek V3.2	27	24	21	15	0	17.5%
Qwen 3.5 Flash	44	14	13	9	7	17.5%
Qwen 3.5 122B	55	19	11	2	0	17.5%
Gemini 3 Flash (Preview)	38	25	22	1	0	17.2%
Hermes 3 70B	52	25	0	0	0	15.3%
Qwen 3.5 Plus (2026-04-20)	37	31	2	0	0	14.0%
Gemma 4 31B	23	18	12	9	5	13.6%
Ministral 3 14B	24	24	7	6	0	12.3%
Qwen 3.6 35B	30	24	4	0	0	11.6%
Gemma 4 31B (Reasoning)	27	13	9	2	0	10.3%
Gemma 4 26B (Reasoning)	16	15	11	7	0	10.1%
Z.AI GLM 4.6	30	19	1	0	0	10.0%
Qwen 3.5 397B A17B	25	12	8	5	0	10.0%
Gemini 3.1 Flash Lite (Reasoning)	15	13	10	5	4	9.5%
Gemma 4 26B	21	18	6	1	0	9.0%
Qwen 3.6 Flash	20	14	4	4	0	8.7%
DeepSeek V3.1	20	13	10	0	0	8.5%
Gemini 2.5 Pro	25	17	0	0	0	8.4%
DeepSeek-V2 Chat	30	10	2	0	0	8.3%
Nemotron 3 Super	18	15	6	2	0	8.1%
DeepSeek V3 (2024-12-26)	16	16	0	0	0	6.3%
Cohere Command R+ (Aug. 2024)	15	7	4	0	0	5.3%
Qwen 2.5 72B	25	0	0	0	0	5.0%
o4 Mini	24	1	0	0	0	4.9%
Llama 3.1 70B	22	0	0	0	0	4.5%
DeepSeek V3 (2025-03-24)	21	0	0	0	0	4.2%
Gemma 3 12B	20	0	0	0	0	4.1%
GPT-OSS 120B	20	0	0	0	0	4.1%
Mistral NeMO	20	0	0	0	0	4.0%
Gemini 2.5 Flash	11	4	3	0	0	3.6%
Mistral Small 3.2 24B	17	0	0	0	0	3.5%
Ministral 8B	7	5	4	0	0	3.3%
Qwen 3.6 27B	10	6	0	0	0	3.1%
Gemini 2.5 Flash Lite	14	0	0	0	0	2.7%
GPT-4o, Aug. 6th (temp=1)	12	0	0	0	0	2.3%
Ministral 3B	4	3	2	0	0	1.9%
Nemotron 3 Nano	9	0	0	0	0	1.8%
Ministral 3 3B	8	0	0	0	0	1.6%
Gemma 3 27B	2	1	0	0	0	0.5%
o4 Mini High	2	0	0	0	0	0.4%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	75	74	74	73	69	72.9%
ByteDance Seed 2.0 Mini	77	76	74	65	55	69.3%
MiniMax M3	81	74	66	64	59	68.8%
Claude Sonnet 5	82	68	63	62	58	66.7%
ByteDance Seed 2.0 Lite	81	75	57	57	52	64.2%
GPT-5.5 (Reasoning, Low)	67	66	64	58	56	62.5%
Claude Sonnet 5 (Reasoning)	86	75	69	44	37	62.2%
Claude Opus 4.6	75	67	63	53	50	61.4%
GPT-5.5	66	65	62	59	52	60.9%
GPT-5.5 (Reasoning)	65	64	62	60	54	60.9%
Claude Opus 4.7	68	63	63	56	52	60.3%
Claude Sonnet 5 (Reasoning, Low)	74	70	57	49	47	59.4%
GPT-5.4 (Reasoning, Low)	70	67	62	51	45	58.8%
Claude Opus 4.8 (Reasoning, Low)	86	69	64	35	33	57.5%
Claude Opus 4.8 (Reasoning)	73	64	60	47	40	56.7%
GPT-5.4 (Reasoning)	63	61	57	55	45	56.3%
Claude Opus 4.7 (Reasoning)	71	67	50	44	38	53.9%
Grok 4.20	70	62	56	48	33	53.8%
GPT-5.4	63	57	52	51	39	52.3%
Claude Sonnet 4.6	63	54	49	49	39	50.6%
MiniMax M2.7	55	54	51	47	46	50.5%
ByteDance Seed 1.6 Flash	61	56	51	48	35	50.4%
Z.AI GLM 5.2 (Reasoning, High)	60	58	50	49	33	50.1%
Grok 4.5 (Reasoning, Low)	58	56	53	51	32	50.0%
GPT-5 Mini	57	54	53	41	37	48.7%
Grok 4.20 (Reasoning)	60	53	46	41	41	48.2%
GPT-5.4 Mini (Reasoning)	67	53	46	43	31	48.2%
Qwen 3.5 9B	67	55	47	42	25	47.2%
MiniMax M2.5	62	54	53	36	30	47.0%
Z.AI GLM 5 Turbo	66	52	50	49	14	46.2%
Claude Sonnet 4.6 (Reasoning)	53	51	50	42	35	46.1%
Grok 4.3	50	47	45	43	42	45.5%
DeepSeek V4 Flash	55	52	52	39	28	45.3%
DeepSeek V4 Pro (Reasoning)	82	56	51	32	0	44.2%
Claude Opus 4.6 (Reasoning)	59	50	41	36	35	44.2%
GPT-5.4 Mini	62	51	46	32	28	43.9%
GPT-5.2	51	47	46	43	32	43.6%
GPT-5.4 Mini (Reasoning, Low)	54	45	40	40	34	42.9%
GPT-5 Nano	46	45	44	41	36	42.4%
Grok 4.5 (Reasoning, High)	57	48	44	43	17	41.7%
GPT-5.1	45	41	40	40	38	40.8%
Grok 4.3 (Reasoning)	57	48	40	28	28	40.0%
Claude Opus 4	58	57	40	36	9	39.9%
Aion 3.0 Mini	67	47	42	24	17	39.5%
Claude Haiku 4.5	55	41	34	34	31	39.1%
Mistral Small 4	58	42	34	33	26	38.6%
Qwen 3.5 122B	54	47	37	32	23	38.5%
GPT-5.4 Nano (Reasoning, Low)	54	50	44	30	14	38.4%
Z.AI GLM 5	50	44	39	34	23	37.8%
GPT-5.4 Nano	44	42	39	31	31	37.4%
Mistral Medium 3.1	54	38	33	30	29	36.9%
Qwen 3.5 27B	49	42	38	34	22	36.8%
DeepSeek V4 Pro	55	44	41	22	21	36.4%
Qwen 3.5 Flash	51	41	35	27	22	35.4%
GPT-4.1	68	35	31	25	17	35.3%
Qwen 3.5 397B A17B	45	41	40	31	17	34.7%
Gemini 3 Flash (Preview)	45	35	35	32	26	34.6%
GPT-5.4 Nano (Reasoning)	47	41	35	26	23	34.4%
Claude Sonnet 4.5	73	46	27	15	2	32.5%
MoonshotAI: Kimi K2.6	53	41	34	19	13	32.0%
Aion 3.0	45	39	26	25	23	31.6%
MoonshotAI: Kimi K2.5	45	41	27	25	16	30.7%
Z.AI GLM 4.7 Flash	43	33	32	27	18	30.4%
Cydonia 24B V4.1	41	40	33	25	5	28.7%
Mistral Small 4 (Reasoning)	44	34	27	22	16	28.5%
Z.AI GLM 4.7	33	31	29	27	20	28.1%
Gemini 3.1 Pro (Preview)	40	35	29	22	14	27.9%
Claude Opus 4.5	52	33	33	12	8	27.6%
Qwen 3 32B	51	46	22	14	0	26.8%
Z.AI GLM 5.1	43	35	23	23	8	26.6%
Gemini 3.5 Flash (Reasoning, Minimal)	43	34	32	24	0	26.6%
Gemini 3 Flash (Preview, Reasoning)	40	31	27	15	10	24.6%
Qwen 3.6 27B	46	24	24	15	13	24.4%
DeepSeek V3.2	37	33	24	19	10	24.3%
Qwen 3.5 35B	34	25	24	21	18	24.3%
Qwen3.6 Max Preview	45	29	29	18	0	24.2%
DeepSeek V3 (2025-03-24)	34	34	32	19	0	23.8%
Qwen 3.6 Flash	38	29	29	13	3	22.5%
Z.AI GLM 4.5	53	37	10	9	0	21.8%
Ministral 3 14B	42	26	24	9	1	20.5%
Gemini 3.5 Flash (Reasoning)	37	26	24	10	5	20.3%
Qwen 2.5 72B	100	0	0	0	0	20.0%
Qwen 3.5 Plus (2026-02-15)	36	30	15	12	5	19.6%
ByteDance Seed 1.6	39	33	9	8	7	19.1%
DeepSeek V4 Flash (Reasoning)	39	22	21	11	2	19.0%
Writer: Palmyra X5	36	30	25	4	0	18.7%
Z.AI GLM 4.6	47	20	16	8	0	18.3%
Qwen3 235B A22B Instruct 2507	40	13	13	12	11	17.6%
Xiaomi MIMO v2.5	46	30	4	4	0	16.9%
Z.AI GLM 4.5 Air	32	22	13	11	7	16.9%
Mistral Large 2	39	15	12	9	0	14.9%
Qwen 3.6 35B	40	19	12	2	0	14.6%
Aion 2.0	24	23	16	10	0	14.5%
Qwen 3.5 Plus (2026-04-20)	42	19	9	1	0	14.3%
Gemini 3.1 Flash Lite	43	10	10	6	0	13.7%
DeepSeek V3.1	29	19	17	0	0	13.0%
Xiaomi MIMO v2.5 Pro	35	28	2	0	0	12.9%
Qwen3.7 Max	21	18	14	0	0	10.5%
Hermes 3 70B	28	14	7	0	0	10.0%
Gemma 3 27B	32	8	0	0	0	8.1%
Gemma 4 31B (Reasoning)	39	2	0	0	0	8.0%
Gemma 4 26B	31	9	0	0	0	8.0%
Mistral Large 3	29	6	4	0	0	7.8%
Gemini 3.1 Flash Lite (Preview)	21	14	0	0	0	6.9%
Claude Sonnet 4	18	13	2	0	0	6.6%
Hermes 3 405B	27	3	0	0	0	5.9%
Gemini 2.5 Pro	13	9	7	0	0	5.8%
Gemma 4 26B (Reasoning)	19	8	0	0	0	5.5%
Gemma 4 31B	17	7	4	0	0	5.4%
Ministral 3B	18	6	0	0	0	5.0%
o4 Mini	12	6	0	0	0	3.8%
Ministral 3 8B	12	3	0	0	0	3.0%
Ministral 8B	6	5	4	0	0	2.9%
GPT-OSS 120B	8	5	0	0	0	2.5%
Gemini 2.5 Flash Lite (Reasoning)	11	0	0	0	0	2.3%
Gemma 3 12B	9	0	0	0	0	1.7%
DeepSeek V3 (2024-12-26)	9	0	0	0	0	1.7%
Cohere Command R+ (Aug. 2024)	8	0	0	0	0	1.6%
Ministral 3 3B	7	0	0	0	0	1.4%
Mistral Small 3.2 24B	7	0	0	0	0	1.3%
Nemotron 3 Super	6	0	0	0	0	1.1%
Gemini 3.1 Flash Lite (Reasoning)	2	0	0	0	0	0.3%
o4 Mini High	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
DeepSeek-V2 Chat	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7 (Reasoning)	100	96	86	79	78	88.0%
Claude Opus 4.7	96	92	82	81	78	85.8%
Claude Opus 4.8 (Reasoning)	96	92	82	79	79	85.8%
GPT-5.5 (Reasoning)	90	89	87	83	80	85.7%
GPT-5.4	92	88	83	82	79	84.8%
MiniMax M3	93	92	88	74	72	83.6%
Claude Opus 4.8 (Reasoning, Low)	100	84	83	78	69	82.8%
Claude Sonnet 5 (Reasoning)	88	86	83	82	73	82.4%
GPT-5.5	90	85	81	81	74	82.2%
Claude Sonnet 4.6 (Reasoning)	87	85	80	80	78	81.7%
ByteDance Seed 2.0 Lite	84	83	82	81	79	81.7%
GPT-5.4 (Reasoning)	86	82	82	82	75	81.5%
ByteDance Seed 1.6 Flash	87	87	85	78	68	81.1%
GPT-5.5 (Reasoning, Low)	85	83	81	78	78	81.0%
Claude Sonnet 5	87	85	80	79	73	80.8%
GPT-5.4 Mini (Reasoning)	87	81	79	78	77	80.4%
ByteDance Seed 2.0 Mini	87	86	79	76	74	80.4%
GPT-5	82	81	80	80	78	80.3%
Z.AI GLM 5 Turbo	92	86	75	74	70	79.6%
GPT-5.4 (Reasoning, Low)	86	83	82	73	73	79.4%
Claude Sonnet 4.6	85	81	76	76	75	78.5%
GPT-5.1	81	79	75	74	73	76.4%
MiniMax M2.7	87	86	82	64	63	76.3%
Claude Opus 4.6 (Reasoning)	81	78	75	75	70	75.9%
Claude Sonnet 5 (Reasoning, Low)	82	82	76	70	68	75.8%
GPT-5.4 Mini (Reasoning, Low)	84	77	75	73	67	75.1%
GPT-5.4 Mini	82	79	73	72	70	75.0%
GPT-5 Mini	80	78	75	75	59	73.5%
Aion 3.0	86	83	75	69	54	73.2%
Grok 4.5 (Reasoning, High)	75	74	73	67	64	70.6%
DeepSeek V4 Pro	84	76	66	66	58	70.0%
MoonshotAI: Kimi K2.6	76	75	68	65	61	69.3%
Grok 4.20	78	76	70	65	54	68.6%
Grok 4.5 (Reasoning, Low)	84	84	62	59	52	68.1%
GPT-5.4 Nano (Reasoning, Low)	71	71	69	66	64	68.1%
Claude Opus 4.6	76	68	66	64	62	67.4%
MiniMax M2.5	73	72	70	68	49	66.5%
Z.AI GLM 5.2 (Reasoning, High)	69	68	68	66	60	66.3%
Claude Haiku 4.5	89	65	63	59	56	66.2%
GPT-5.2	76	70	68	66	48	65.4%
DeepSeek V4 Flash (Reasoning)	78	69	65	62	48	64.4%
ByteDance Seed 1.6	69	69	64	63	55	63.9%
GPT-5.4 Nano (Reasoning)	69	67	62	61	59	63.8%
Aion 3.0 Mini	70	69	67	58	53	63.5%
Grok 4.20 (Reasoning)	68	65	62	61	61	63.4%
GPT-5.4 Nano	66	65	62	61	59	62.6%
MoonshotAI: Kimi K2.5	72	71	62	54	54	62.3%
Claude Opus 4.5	69	68	64	58	51	62.0%
Z.AI GLM 5.1	72	67	67	52	49	61.5%
Claude Sonnet 4.5	71	64	64	57	51	61.4%
Gemini 3.5 Flash (Reasoning, Minimal)	69	65	63	58	52	61.4%
Xiaomi MIMO v2.5	84	83	72	34	34	61.3%
Qwen 3 32B	73	64	63	57	48	61.0%
Writer: Palmyra X5	68	63	61	57	55	60.9%
DeepSeek V4 Flash	74	68	61	54	46	60.9%
Ministral 3 8B	82	71	59	44	44	60.0%
Grok 4.3	73	68	54	52	51	59.6%
Claude Opus 4	64	61	58	57	54	59.0%
GPT-4.1	69	64	61	61	39	58.7%
Aion 2.0	68	67	60	48	45	57.7%
DeepSeek V4 Pro (Reasoning)	69	63	60	57	39	57.4%
Z.AI GLM 5	65	62	59	54	46	57.1%
Gemini 3.5 Flash (Reasoning)	70	63	57	54	37	56.2%
Z.AI GLM 4.7	71	56	55	54	43	55.8%
Ministral 8B	72	64	57	46	37	55.4%
GPT-5 Nano	70	63	49	46	42	54.0%
Gemini 3 Flash (Preview)	60	57	56	49	47	53.7%
Qwen 3.6 Flash	68	63	52	46	38	53.4%
Qwen3 235B A22B Instruct 2507	79	61	42	42	41	52.7%
Mistral Small 4 (Reasoning)	78	51	48	46	39	52.4%
Qwen3.6 Max Preview	67	56	51	48	39	52.1%
Mistral Small 4	62	56	56	50	35	51.8%
Ministral 3 14B	58	53	51	47	47	51.3%
DeepSeek V3 (2025-03-24)	78	58	56	47	16	51.3%
Grok 4.3 (Reasoning)	60	58	54	48	33	50.5%
Gemini 2.5 Pro	57	55	47	44	42	49.0%
Qwen 3.5 9B	76	65	46	34	23	48.8%
DeepSeek V3.2	56	56	53	41	35	48.1%
Qwen 3.5 Plus (2026-04-20)	56	53	47	43	39	47.6%
Mistral Medium 3.1	68	61	49	30	29	47.4%
Qwen 3.6 27B	63	48	44	38	37	46.0%
Z.AI GLM 4.5	76	56	38	35	24	45.7%
Claude Sonnet 4	57	48	44	42	35	45.2%
Gemini 3 Flash (Preview, Reasoning)	62	53	51	38	22	45.1%
Mistral Large 2	69	57	45	32	20	44.4%
Z.AI GLM 4.7 Flash	53	51	46	39	32	44.3%
Qwen 3.5 Flash	55	44	43	40	38	44.3%
Z.AI GLM 4.6	52	47	44	38	36	43.5%
Gemini 3.1 Flash Lite (Reasoning)	56	47	42	37	29	42.4%
Cydonia 24B V4.1	55	52	48	29	26	41.8%
o4 Mini	65	50	48	31	14	41.7%
Gemini 3.1 Pro (Preview)	50	50	49	33	25	41.4%
DeepSeek V3.1	48	46	40	40	32	41.3%
Mistral Large 3	48	46	45	44	21	41.0%
Qwen 3.5 397B A17B	61	51	36	25	24	39.4%
Qwen 3.5 122B	54	49	45	32	12	38.5%
Qwen3.7 Max	43	39	37	36	36	38.3%
Xiaomi MIMO v2.5 Pro	55	41	35	31	29	38.1%
Qwen 3.5 35B	51	39	37	33	13	34.5%
Gemini 3.1 Flash Lite (Preview)	51	37	31	27	27	34.5%
Qwen 3.5 Plus (2026-02-15)	52	45	35	21	19	34.4%
o4 Mini High	47	43	35	25	22	34.2%
Z.AI GLM 4.5 Air	57	41	38	33	0	33.7%
Qwen 3.5 27B	48	47	26	22	22	33.1%
Qwen 3.6 35B	49	40	30	30	13	32.4%
Gemma 4 31B (Reasoning)	47	34	23	21	18	28.6%
DeepSeek V3 (2024-12-26)	56	50	25	10	0	28.3%
Hermes 3 405B	57	37	24	11	10	27.8%
Gemma 4 31B	39	36	30	21	7	26.8%
Nemotron 3 Super	32	30	25	23	15	25.1%
Gemma 3 27B	59	29	19	17	0	24.8%
GPT-OSS 120B	48	26	23	8	5	21.9%
Llama 3.1 70B	60	35	13	0	0	21.7%
Ministral 3 3B	53	33	15	3	0	20.7%
Ministral 3B	31	28	27	17	0	20.6%
Hermes 3 70B	36	35	25	5	0	20.3%
Gemma 3 12B	25	23	22	20	11	20.2%
Gemma 4 26B	41	26	22	7	6	20.2%
Gemini 2.5 Flash Lite (Reasoning)	43	33	15	6	0	19.4%
Gemini 3.1 Flash Lite	28	28	19	15	5	19.0%
GPT-4.1 Mini	37	28	14	8	0	17.3%
Mistral Small 3.2 24B	43	41	0	0	0	16.8%
DeepSeek-V2 Chat	60	11	9	3	0	16.3%
Gemini 2.5 Flash Lite	38	23	8	6	0	15.1%
Gemma 4 26B (Reasoning)	35	23	14	3	0	15.1%
Mistral NeMO	31	24	17	1	0	14.6%
Gemini 2.5 Flash	27	23	21	3	0	14.6%
WizardLM 2 8x22b	38	29	0	0	0	13.4%
Gemini 2.5 Flash (Reasoning)	25	17	16	6	0	12.6%
Nemotron 3 Nano	35	16	8	2	1	12.3%
Arcee AI: Trinity Mini	25	24	0	0	0	9.8%
Cohere Command R+ (Aug. 2024)	17	12	7	3	0	7.8%
Gemma 3 4B	17	9	8	0	0	6.8%
GPT-4.1 Nano	33	0	0	0	0	6.5%
Inception Mercury 2	15	2	0	0	0	3.4%
Qwen 2.5 72B	8	1	0	0	0	1.9%
GPT-4o Mini (temp=1)	3	0	0	0	0	0.6%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 5 (Reasoning, Low)	95	95	95	91	78	91.0%
Claude Sonnet 5	91	88	87	82	78	85.2%
Claude Opus 4.7 (Reasoning)	96	87	82	82	79	85.2%
Claude Opus 4.7	96	91	84	79	74	84.9%
GPT-5.4 Mini (Reasoning)	91	85	83	81	80	83.9%
GPT-5.4	92	89	83	78	76	83.7%
Claude Opus 4.8 (Reasoning, Low)	92	88	82	79	77	83.6%
MiniMax M3	86	85	84	84	77	83.1%
Claude Sonnet 5 (Reasoning)	91	86	82	77	76	82.5%
GPT-5.5 (Reasoning, Low)	85	83	82	82	74	81.2%
GPT-5.4 (Reasoning)	89	83	81	78	75	81.2%
Claude Sonnet 4.6 (Reasoning)	90	87	83	74	71	80.9%
GPT-5.5	85	80	80	79	78	80.4%
Aion 3.0 Mini	88	87	82	72	71	80.0%
GPT-5.4 Mini	84	84	82	79	70	80.0%
Claude Sonnet 4.6	91	78	77	76	74	79.3%
GPT-5.4 (Reasoning, Low)	82	82	81	79	72	79.2%
ByteDance Seed 2.0 Lite	85	81	81	77	71	79.1%
GPT-5.5 (Reasoning)	82	80	78	77	76	78.7%
GPT-5 Mini	85	80	79	76	72	78.4%
Claude Opus 4.8 (Reasoning)	82	82	80	74	67	77.1%
GPT-5.4 Mini (Reasoning, Low)	91	85	81	69	55	76.0%
GPT-5	85	76	75	72	68	75.2%
Claude Opus 4.5	89	81	70	69	67	75.0%
Aion 3.0	91	79	77	73	54	75.0%
Claude Opus 4.6	83	77	76	72	66	74.7%
DeepSeek V4 Flash (Reasoning)	81	75	71	70	67	72.6%
Claude Opus 4.6 (Reasoning)	87	81	74	61	59	72.3%
GPT-5.2	77	73	73	72	64	71.6%
Z.AI GLM 5.2 (Reasoning, High)	81	79	67	66	64	71.5%
Grok 4.5 (Reasoning, Low)	81	80	69	66	61	71.4%
DeepSeek V4 Pro	75	72	70	68	66	70.1%
Grok 4.5 (Reasoning, High)	77	73	71	66	62	70.0%
DeepSeek V4 Pro (Reasoning)	81	77	69	65	57	69.9%
ByteDance Seed 2.0 Mini	81	77	65	63	49	67.1%
GPT-5.1	72	72	69	64	57	66.8%
GPT-5.4 Nano (Reasoning, Low)	79	68	67	63	56	66.7%
ByteDance Seed 1.6	78	72	67	60	56	66.6%
ByteDance Seed 1.6 Flash	80	68	64	61	60	66.6%
Z.AI GLM 4.5	78	75	66	58	56	66.3%
Xiaomi MIMO v2.5	81	77	75	50	47	66.3%
Z.AI GLM 5.1	82	67	66	65	47	65.3%
GPT-5.4 Nano	74	73	66	59	52	64.8%
DeepSeek V4 Flash	74	69	64	59	57	64.5%
Z.AI GLM 5	75	68	63	61	54	64.2%
GPT-5.4 Nano (Reasoning)	67	65	63	63	61	63.7%
MoonshotAI: Kimi K2.5	73	66	62	56	53	61.9%
Claude Haiku 4.5	79	74	66	48	43	61.9%
MoonshotAI: Kimi K2.6	71	71	70	53	45	61.8%
GPT-5 Nano	70	61	61	59	56	61.4%
Grok 4.20 (Reasoning)	74	67	62	53	50	60.8%
Z.AI GLM 5 Turbo	68	64	61	57	53	60.6%
Grok 4.20	68	63	62	62	47	60.4%
Claude Opus 4	75	64	62	56	46	60.4%
MiniMax M2.7	80	71	59	48	43	60.3%
MiniMax M2.5	73	69	62	48	47	59.9%
Qwen3.7 Max	74	68	54	51	50	59.1%
Grok 4.3	83	62	51	51	48	58.9%
Claude Sonnet 4.5	82	63	48	47	46	57.1%
Xiaomi MIMO v2.5 Pro	80	66	63	52	24	57.0%
Qwen 3.5 397B A17B	63	61	56	51	51	56.3%
Gemini 3.5 Flash (Reasoning, Minimal)	77	73	57	37	36	56.0%
Z.AI GLM 4.5 Air	81	65	55	45	34	56.0%
Gemini 3.1 Pro (Preview)	65	65	63	44	39	55.2%
Qwen 3.5 Flash	64	62	60	45	44	54.9%
DeepSeek-V2 Chat	76	55	55	51	33	54.2%
Qwen 3.6 27B	68	65	58	49	30	54.0%
Grok 4.3 (Reasoning)	60	60	57	49	43	54.0%
Qwen 3.6 Flash	66	60	59	47	36	53.5%
Qwen3 235B A22B Instruct 2507	81	72	53	46	14	53.0%
Gemini 3.5 Flash (Reasoning)	60	56	56	47	45	52.9%
Claude Sonnet 4	62	61	56	50	35	52.8%
Qwen 3.5 9B	66	57	52	46	41	52.4%
Qwen 3.5 122B	57	53	52	50	46	51.5%
Hermes 3 405B	79	52	50	40	35	51.2%
Ministral 3 8B	70	70	62	34	21	51.2%
Hermes 3 70B	75	67	39	37	36	50.9%
Gemini 3.1 Flash Lite (Preview)	71	65	47	39	29	50.2%
Qwen 3.5 27B	66	53	50	49	31	49.4%
Gemini 3 Flash (Preview)	53	51	51	44	44	48.8%
Qwen 3.6 35B	66	56	48	40	34	48.6%
Ministral 3 14B	65	65	42	37	30	47.7%
Z.AI GLM 4.7 Flash	54	51	50	41	39	47.1%
Qwen3.6 Max Preview	59	55	50	39	33	47.1%
Mistral Large 2	65	60	47	38	25	46.9%
Qwen 3.5 Plus (2026-02-15)	60	59	52	34	29	46.9%
Gemini 2.5 Pro	54	52	48	46	33	46.8%
Gemma 4 31B (Reasoning)	53	52	50	39	37	46.3%
Mistral Large 3	57	54	54	41	27	46.3%
Mistral Medium 3.1	60	53	45	39	34	46.2%
Writer: Palmyra X5	59	52	48	39	32	46.0%
Gemma 4 31B	55	48	48	43	36	46.0%
Qwen 3.5 35B	51	50	45	41	38	45.1%
Z.AI GLM 4.7	56	50	47	40	31	44.7%
Qwen 3.5 Plus (2026-04-20)	54	49	40	39	38	44.0%
Qwen 3 32B	65	47	41	35	30	43.6%
Cydonia 24B V4.1	64	57	48	44	0	42.6%
Gemini 3 Flash (Preview, Reasoning)	51	48	42	36	35	42.5%
DeepSeek V3.2	58	50	40	34	30	42.3%
Z.AI GLM 4.6	59	51	43	36	17	41.2%
Aion 2.0	53	50	47	29	27	41.1%
Mistral Small 4 (Reasoning)	58	49	43	38	17	40.9%
Gemini 2.5 Flash	56	51	43	36	14	39.9%
DeepSeek V3.1	54	49	41	41	13	39.6%
Gemini 3.1 Flash Lite	56	55	46	22	17	39.2%
GPT-4.1	59	47	38	28	22	38.8%
Gemma 4 26B	51	50	37	33	17	37.6%
Ministral 8B	54	48	27	27	26	36.7%
Ministral 3B	52	45	35	30	20	36.4%
Gemini 2.5 Flash (Reasoning)	56	40	39	26	11	34.4%
Gemini 3.1 Flash Lite (Reasoning)	39	36	36	31	23	33.1%
DeepSeek V3 (2025-03-24)	39	36	33	32	16	31.2%
Nemotron 3 Nano	53	39	26	22	13	30.7%
Mistral Small 4	51	32	26	25	20	30.6%
Gemma 3 4B	54	38	32	13	13	29.8%
DeepSeek V3 (2024-12-26)	54	49	27	12	7	29.8%
Llama 3.1 70B	66	64	15	0	0	29.1%
GPT-4.1 Mini	63	33	32	14	0	28.5%
Gemini 2.5 Flash Lite (Reasoning)	47	39	36	14	0	27.1%
Cohere Command R+ (Aug. 2024)	64	60	10	0	0	26.9%
Gemma 3 27B	41	39	28	19	2	25.9%
o4 Mini	36	32	30	30	1	25.7%
Gemma 4 26B (Reasoning)	32	26	25	21	20	24.7%
Nemotron 3 Super	31	31	28	17	11	23.5%
Ministral 3 3B	38	31	25	15	0	21.9%
Gemma 3 12B	57	20	17	14	1	21.7%
Arcee AI: Trinity Mini	39	26	25	13	0	20.5%
GPT-OSS 120B	26	19	19	18	13	19.0%
Mistral NeMO	29	23	22	14	7	18.9%
o4 Mini High	38	30	17	8	0	18.7%
GPT-4o Mini (temp=1)	36	32	15	0	0	16.5%
Inception Mercury 2	32	22	20	7	0	16.3%
Gemini 2.5 Flash Lite	39	22	13	5	1	15.9%
Mistral Small 3.2 24B	29	21	20	3	0	14.7%
GPT-4.1 Nano	29	14	14	11	0	13.6%
Qwen 2.5 72B	27	22	11	9	0	13.6%
WizardLM 2 8x22b	29	21	17	0	0	13.4%
GPT-4o, Aug. 6th (temp=0)	25	19	9	3	0	11.1%
GPT-4o, Aug. 6th (temp=1)	21	12	6	1	0	8.0%
GPT-4o Mini (temp=0)	13	1	0	0	0	2.9%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7	95	87	86	86	74	85.6%
Claude Sonnet 4.6	91	91	90	88	63	84.9%
ByteDance Seed 1.6 Flash	91	87	85	83	76	84.4%
ByteDance Seed 2.0 Lite	91	86	81	79	77	82.9%
Claude Opus 4.8 (Reasoning, Low)	87	87	84	79	69	81.0%
Claude Sonnet 4.6 (Reasoning)	94	82	82	73	71	80.3%
ByteDance Seed 2.0 Mini	88	84	82	81	66	80.1%
MiniMax M3	87	83	82	73	71	79.0%
Claude Sonnet 5 (Reasoning)	92	88	82	69	55	77.2%
Claude Sonnet 5	87	86	76	69	67	76.9%
Claude Opus 4.8 (Reasoning)	82	78	77	76	71	76.9%
GPT-5	81	78	77	76	71	76.6%
DeepSeek V4 Flash	93	83	80	72	49	75.4%
GPT-5 Mini	81	78	73	73	69	74.9%
Claude Opus 4.7 (Reasoning)	92	81	73	69	54	74.1%
Claude Sonnet 5 (Reasoning, Low)	84	82	75	65	63	73.8%
Z.AI GLM 5.2 (Reasoning, High)	84	76	73	71	64	73.5%
Claude Opus 4.6	84	76	70	70	64	72.7%
GPT-5.4 (Reasoning, Low)	78	77	73	70	61	72.1%
GPT-5.4 (Reasoning)	77	73	72	70	68	71.8%
GPT-5.5	79	74	73	73	59	71.6%
GPT-5.4	78	75	69	67	66	70.9%
Claude Opus 4.6 (Reasoning)	76	76	72	68	62	70.9%
GPT-5.5 (Reasoning)	86	71	70	66	61	70.8%
MiniMax M2.7	80	77	70	66	59	70.6%
ByteDance Seed 1.6	91	75	68	65	53	70.4%
DeepSeek V4 Flash (Reasoning)	77	75	70	67	62	70.2%
GPT-5.5 (Reasoning, Low)	76	72	70	70	61	69.6%
DeepSeek V4 Pro	82	75	69	67	52	69.0%
GPT-5.1	88	70	67	60	59	68.8%
Z.AI GLM 5.1	78	76	72	69	49	68.6%
DeepSeek V4 Pro (Reasoning)	73	73	72	66	59	68.6%
Grok 4.5 (Reasoning, High)	76	71	69	66	59	68.3%
Z.AI GLM 5 Turbo	82	73	64	61	58	67.6%
Aion 3.0 Mini	76	71	70	62	56	66.9%
Aion 3.0	76	76	68	60	54	66.7%
Grok 4.3	74	68	67	67	57	66.6%
GPT-5.4 Mini	74	71	70	56	53	64.9%
Claude Haiku 4.5	71	70	68	62	51	64.4%
MiniMax M2.5	74	71	69	57	49	64.0%
GPT-5.4 Mini (Reasoning)	73	67	66	60	52	63.5%
Claude Opus 4.5	71	70	64	62	47	62.7%
Claude Sonnet 4.5	77	68	64	63	39	62.4%
GPT-5.4 Nano	66	66	64	63	53	62.4%
Grok 4.5 (Reasoning, Low)	70	64	63	59	55	62.4%
GPT-5.4 Nano (Reasoning)	79	65	57	56	53	62.0%
GPT-5.4 Nano (Reasoning, Low)	79	70	57	52	50	61.7%
Writer: Palmyra X5	72	71	64	54	44	61.1%
GPT-5.4 Mini (Reasoning, Low)	72	64	59	56	54	61.1%
Claude Opus 4	75	60	60	59	48	60.3%
Z.AI GLM 5	73	63	55	55	53	60.0%
GPT-5.2	66	64	62	59	49	60.0%
Xiaomi MIMO v2.5	72	71	62	52	43	59.9%
MoonshotAI: Kimi K2.6	71	69	64	53	39	59.1%
Grok 4.20	65	61	60	55	53	58.7%
Grok 4.20 (Reasoning)	78	55	53	53	48	57.5%
DeepSeek V3 (2025-03-24)	100	59	47	41	39	57.1%
Claude Sonnet 4	82	58	57	54	35	57.1%
GPT-5 Nano	61	59	59	48	48	55.0%
Gemini 3.5 Flash (Reasoning, Minimal)	63	56	53	51	49	54.3%
Qwen 3.5 397B A17B	66	64	58	48	35	54.0%
Mistral Large 3	69	69	52	42	40	54.0%
Qwen3 235B A22B Instruct 2507	65	55	50	49	43	52.7%
Grok 4.3 (Reasoning)	60	59	56	54	34	52.5%
GPT-4.1	62	61	49	40	38	50.1%
Qwen 3 32B	66	64	52	35	33	50.0%
Ministral 3 14B	65	64	56	33	32	49.9%
Z.AI GLM 4.7	57	56	52	46	38	49.7%
Aion 2.0	60	58	56	38	37	49.5%
Qwen 3.5 Flash	61	59	58	40	29	49.5%
Gemini 3 Flash (Preview, Reasoning)	70	62	44	39	28	48.9%
Qwen3.6 Max Preview	67	55	46	40	34	48.5%
Qwen 3.5 122B	59	50	49	45	39	48.3%
Z.AI GLM 4.5	59	55	53	41	33	48.1%
Mistral Large 2	60	51	51	46	31	47.8%
Gemini 3 Flash (Preview)	67	49	47	45	31	47.7%
MoonshotAI: Kimi K2.5	57	56	48	44	33	47.6%
Qwen 3.6 27B	54	52	51	41	40	47.6%
Mistral Small 4 (Reasoning)	65	59	41	39	33	47.3%
Ministral 3 8B	71	65	40	34	26	47.2%
Qwen 3.5 9B	55	52	46	43	38	46.9%
Mistral Small 4	58	54	52	37	33	46.8%
Qwen 3.5 35B	69	57	42	31	29	45.4%
Qwen 3.5 Plus (2026-02-15)	59	50	44	38	33	45.1%
Hermes 3 70B	68	49	43	37	24	44.2%
Z.AI GLM 4.7 Flash	57	46	41	39	37	43.8%
Mistral Medium 3.1	49	48	47	45	30	43.7%
DeepSeek V3.2	53	49	45	38	26	42.1%
Gemini 3.5 Flash (Reasoning)	60	45	41	30	30	41.3%
Xiaomi MIMO v2.5 Pro	70	53	34	23	21	40.3%
Cydonia 24B V4.1	51	51	48	37	12	39.6%
Gemini 3.1 Flash Lite (Reasoning)	56	41	40	34	27	39.4%
Qwen 3.5 27B	50	46	43	37	19	39.1%
Gemma 4 31B	48	44	37	32	32	38.9%
Gemini 3.1 Pro (Preview)	47	42	41	36	24	37.9%
Qwen 3.6 35B	52	41	40	33	19	37.2%
Gemma 4 31B (Reasoning)	43	42	40	31	21	35.4%
Qwen3.7 Max	50	39	31	26	24	34.1%
Z.AI GLM 4.6	52	40	29	27	19	33.4%
Gemini 3.1 Flash Lite (Preview)	47	43	38	20	17	33.1%
Qwen 3.6 Flash	54	43	32	23	13	33.0%
Qwen 3.5 Plus (2026-04-20)	48	39	38	28	12	32.9%
DeepSeek V3 (2024-12-26)	54	44	38	15	4	31.3%
Gemma 4 26B (Reasoning)	41	35	31	24	21	30.5%
DeepSeek V3.1	49	31	31	24	16	30.3%
DeepSeek-V2 Chat	47	37	30	24	12	29.9%
Ministral 3 3B	47	40	30	18	14	29.8%
Gemini 2.5 Pro	47	34	24	21	17	28.8%
Hermes 3 405B	54	48	18	9	9	27.4%
Z.AI GLM 4.5 Air	41	30	25	16	15	25.6%
Ministral 3B	48	29	18	17	12	25.1%
Ministral 8B	44	30	28	21	0	24.8%
Cohere Command R+ (Aug. 2024)	69	37	18	0	0	24.8%
o4 Mini High	35	28	26	19	16	24.7%
Mistral NeMO	55	28	17	12	12	24.7%
Gemini 3.1 Flash Lite	59	27	18	13	4	24.3%
Nemotron 3 Super	36	34	26	17	4	23.4%
o4 Mini	35	26	20	20	15	23.4%
Gemma 4 26B	47	30	16	15	4	22.2%
GPT-4.1 Mini	60	42	1	0	0	20.7%
Gemma 3 27B	50	26	14	8	1	19.9%
Mistral Small 3.2 24B	45	24	21	1	0	18.1%
Arcee AI: Trinity Mini	54	23	10	0	0	17.5%
Llama 3.1 70B	63	24	0	0	0	17.5%
Gemini 2.5 Flash	32	23	14	11	4	16.7%
GPT-OSS 120B	25	17	14	14	7	15.6%
Gemini 2.5 Flash Lite	30	18	11	0	0	11.7%
Qwen 2.5 72B	47	10	0	0	0	11.5%
Nemotron 3 Nano	21	16	9	3	1	10.0%
Gemini 2.5 Flash (Reasoning)	24	18	3	0	0	9.1%
Inception Mercury 2	26	17	2	0	0	8.8%
GPT-4.1 Nano	18	16	10	0	0	8.8%
GPT-4o, Aug. 6th (temp=1)	23	12	3	1	0	7.8%
Gemma 3 4B	20	4	0	0	0	4.8%
Gemma 3 12B	16	7	0	0	0	4.5%
Gemini 2.5 Flash Lite (Reasoning)	9	6	5	0	0	4.0%
GPT-4o, Aug. 6th (temp=0)	15	0	0	0	0	3.1%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.7	100	95	86	85	76	88.5%
Claude Sonnet 5 (Reasoning)	91	91	79	77	77	83.1%
Claude Sonnet 4.6	100	85	77	77	76	83.0%
Claude Sonnet 5	87	87	83	82	72	82.4%
ByteDance Seed 2.0 Lite	92	90	87	73	70	82.4%
Claude Opus 4.7 (Reasoning)	95	91	80	72	67	81.1%
MiniMax M3	92	85	75	70	70	78.4%
Claude Opus 4.6 (Reasoning)	88	82	75	72	70	77.7%
GPT-5.5 (Reasoning, Low)	85	78	77	74	73	77.4%
GPT-5	80	80	79	73	72	77.0%
GPT-5.5 (Reasoning)	83	78	78	76	70	76.9%
GPT-5.4 (Reasoning, Low)	86	82	77	71	67	76.8%
GPT-5.4	84	81	73	72	71	76.2%
Claude Opus 4.8 (Reasoning, Low)	95	83	73	67	59	75.7%
GPT-5.4 Mini (Reasoning, Low)	79	77	76	73	69	74.7%
GPT-5.4 (Reasoning)	82	78	76	71	64	74.5%
Claude Sonnet 4.6 (Reasoning)	81	78	74	69	69	74.4%
MiniMax M2.5	81	76	74	70	66	73.6%
GPT-5.4 Mini	79	75	70	70	67	72.2%
MiniMax M2.7	87	74	72	66	63	72.2%
GPT-5.5	83	79	68	68	63	72.1%
GPT-5 Mini	83	75	69	66	66	72.0%
GPT-5.4 Mini (Reasoning)	80	74	73	68	64	71.9%
Claude Sonnet 5 (Reasoning, Low)	82	80	79	61	53	71.0%
Claude Opus 4.5	77	73	69	67	67	70.4%
ByteDance Seed 2.0 Mini	75	74	71	66	61	69.3%
Z.AI GLM 5 Turbo	84	76	64	61	60	69.0%
Grok 4.5 (Reasoning, High)	83	71	70	60	60	69.0%
Claude Haiku 4.5	77	74	70	66	55	68.5%
GPT-5.4 Nano (Reasoning, Low)	72	72	66	66	63	67.6%
Claude Opus 4	88	72	62	61	55	67.6%
ByteDance Seed 1.6 Flash	88	81	71	50	48	67.5%
Claude Sonnet 4.5	78	71	64	63	60	67.3%
Aion 3.0	81	68	66	60	58	66.6%
Z.AI GLM 5.1	82	72	70	59	51	66.6%
ByteDance Seed 1.6	85	69	68	60	47	65.9%
GPT-5.4 Nano	72	67	66	61	60	65.1%
Claude Opus 4.8 (Reasoning)	75	74	63	63	51	65.0%
Grok 4.5 (Reasoning, Low)	71	70	68	63	50	64.3%
GPT-5.1	74	68	68	57	55	64.3%
Cydonia 24B V4.1	81	70	69	63	37	64.0%
Aion 3.0 Mini	86	68	55	55	53	63.3%
Z.AI GLM 5	81	68	67	56	44	63.2%
GPT-5.4 Nano (Reasoning)	79	63	60	57	56	63.0%
Z.AI GLM 4.7	77	74	64	54	39	61.5%
WizardLM 2 8x22b	100	100	66	28	10	60.8%
Gemini 3.5 Flash (Reasoning)	67	67	63	55	50	60.3%
Xiaomi MIMO v2.5 Pro	91	78	58	47	27	60.2%
GPT-5.2	79	66	61	57	36	59.8%
Claude Sonnet 4	75	71	59	48	45	59.4%
DeepSeek V4 Flash	66	62	61	55	51	59.2%
DeepSeek V4 Flash (Reasoning)	70	59	57	55	53	58.9%
DeepSeek V4 Pro	71	69	65	47	41	58.7%
Claude Opus 4.6	68	61	57	55	47	57.8%
DeepSeek V3 (2025-03-24)	83	57	55	54	38	57.5%
Z.AI GLM 5.2 (Reasoning, High)	67	57	56	54	52	57.3%
MoonshotAI: Kimi K2.5	77	61	60	46	41	57.0%
DeepSeek V4 Pro (Reasoning)	66	63	59	49	40	55.4%
Grok 4.20	67	63	62	56	28	55.2%
Gemini 3 Flash (Preview, Reasoning)	74	63	56	45	36	54.8%
MoonshotAI: Kimi K2.6	70	62	54	41	40	53.5%
Qwen 3.5 122B	63	56	55	46	44	52.8%
Gemini 3 Flash (Preview)	67	57	52	43	43	52.3%
Z.AI GLM 4.5	72	59	51	41	35	51.6%
Gemini 3.5 Flash (Reasoning, Minimal)	63	53	52	43	42	50.6%
Z.AI GLM 4.7 Flash	68	56	48	46	33	50.2%
Grok 4.20 (Reasoning)	71	50	49	45	32	49.4%
Qwen 3.5 Flash	69	55	55	35	32	49.4%
Qwen 3.6 Flash	60	57	52	46	29	48.8%
Qwen 3.5 397B A17B	59	57	46	40	40	48.6%
DeepSeek-V2 Chat	74	50	46	39	33	48.5%
Ministral 3 14B	61	56	48	44	31	48.1%
Hermes 3 70B	67	61	47	37	29	48.1%
Qwen3.7 Max	53	52	49	42	40	47.3%
Mistral Small 4 (Reasoning)	65	51	48	43	30	47.1%
Qwen 3.5 9B	61	52	42	41	38	46.7%
Aion 2.0	64	62	41	38	29	46.5%
Mistral Large 3	70	58	52	28	23	46.2%
GPT-5 Nano	55	47	45	42	39	45.7%
Qwen 3.5 35B	57	49	45	37	33	44.1%
Xiaomi MIMO v2.5	51	50	47	44	25	43.6%
DeepSeek V3.1	61	54	37	35	29	43.2%
Gemma 4 31B	48	47	45	36	35	42.1%
DeepSeek V3.2	50	49	47	39	25	42.0%
Gemini 3.1 Flash Lite (Preview)	57	52	44	32	25	41.9%
Qwen3 235B A22B Instruct 2507	59	57	36	34	22	41.7%
Gemini 3.1 Pro (Preview)	51	51	43	36	26	41.7%
Qwen 3.6 35B	57	45	37	35	31	41.0%
Qwen 3.5 Plus (2026-02-15)	51	48	37	35	31	40.3%
Writer: Palmyra X5	61	53	51	26	10	40.3%
Mistral Small 4	54	45	37	33	33	40.3%
Mistral Medium 3.1	50	46	37	34	30	39.5%
Mistral Large 2	57	46	43	32	17	38.8%
Ministral 3 8B	50	46	37	33	27	38.6%
Qwen 3.5 Plus (2026-04-20)	48	42	38	34	27	37.9%
Qwen 3.5 27B	47	44	43	40	14	37.5%
Gemini 2.5 Pro	61	49	31	26	17	36.8%
Grok 4.3	61	45	44	31	0	36.1%
Qwen 3.6 27B	46	45	45	42	0	35.7%
Gemma 4 31B (Reasoning)	58	37	34	24	24	35.5%
Qwen3.6 Max Preview	59	33	30	24	23	33.8%
DeepSeek V3 (2024-12-26)	62	37	37	25	6	33.6%
Qwen 3 32B	50	43	41	30	0	32.9%
Ministral 8B	50	46	32	22	11	32.2%
Gemini 3.1 Flash Lite (Reasoning)	50	44	28	23	15	32.1%
Z.AI GLM 4.6	49	40	31	21	17	31.5%
Gemma 3 27B	38	37	32	29	19	30.9%
Grok 4.3 (Reasoning)	62	50	38	0	0	30.1%
Gemini 3.1 Flash Lite	37	35	33	30	14	30.0%
Gemma 3 4B	51	47	29	10	8	29.1%
Gemma 4 26B (Reasoning)	45	35	29	28	2	27.8%
o4 Mini	48	43	27	21	0	27.6%
GPT-4.1	42	38	23	22	10	27.4%
o4 Mini High	40	30	26	21	17	26.7%
Hermes 3 405B	61	44	27	0	0	26.3%
Gemini 2.5 Flash	46	41	29	13	0	25.7%
Ministral 3B	60	32	28	4	0	24.7%
Gemma 4 26B	32	31	24	23	7	23.5%
Z.AI GLM 4.5 Air	35	34	23	18	7	23.4%
Gemini 2.5 Flash Lite	60	32	12	11	0	23.1%
Llama 3.1 70B	37	32	21	18	0	21.6%
Ministral 3 3B	58	23	11	8	0	20.0%
Cohere Command R+ (Aug. 2024)	52	12	7	0	0	14.2%
Gemini 2.5 Flash Lite (Reasoning)	30	19	10	4	0	12.5%
Arcee AI: Trinity Mini	19	17	13	0	0	9.9%
Gemini 2.5 Flash (Reasoning)	22	15	12	0	0	9.8%
Gemma 3 12B	23	13	6	2	0	8.9%
Nemotron 3 Super	22	10	9	0	0	8.2%
GPT-4.1 Nano	25	14	2	0	0	8.1%
Mistral NeMO	29	3	2	1	0	7.1%
Qwen 2.5 72B	27	0	0	0	0	5.5%
GPT-OSS 120B	21	0	0	0	0	4.2%
Inception Mercury 2	17	0	0	0	0	3.3%
GPT-4o, Aug. 6th (temp=0)	15	0	0	0	0	3.0%
GPT-4.1 Mini	7	6	0	0	0	2.7%
GPT-4o, Aug. 6th (temp=1)	11	0	0	0	0	2.3%
Nemotron 3 Nano	4	2	0	0	0	1.2%
GPT-4o Mini (temp=1)	5	0	0	0	0	1.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.5	79	77	77	74	63	73.7%
GPT-5.4 (Reasoning)	77	74	69	68	68	71.1%
Claude Opus 4.7	71	70	70	66	66	68.5%
Claude Sonnet 5 (Reasoning)	76	74	73	61	55	67.7%
GPT-5.5 (Reasoning)	74	68	67	66	63	67.4%
ByteDance Seed 2.0 Lite	79	71	64	62	59	67.0%
GPT-5.5 (Reasoning, Low)	75	71	66	61	56	65.7%
GPT-5.4 (Reasoning, Low)	79	66	65	56	55	64.4%
Claude Sonnet 4.6 (Reasoning)	78	64	61	59	51	62.8%
Claude Opus 4.7 (Reasoning)	78	73	63	53	46	62.4%
Claude Opus 4.8 (Reasoning)	81	76	67	56	31	62.1%
GPT-5.4 Mini	75	68	57	56	54	62.1%
GPT-5.4 Mini (Reasoning)	69	63	63	59	55	61.9%
GPT-5.4	74	69	63	52	51	61.8%
ByteDance Seed 1.6 Flash	65	64	62	58	51	60.1%
MiniMax M3	79	63	60	50	46	59.6%
GPT-5.4 Mini (Reasoning, Low)	65	61	58	58	54	59.4%
GPT-5	73	68	63	54	39	59.3%
Claude Sonnet 5	74	68	50	46	41	56.0%
ByteDance Seed 2.0 Mini	75	68	55	46	33	55.6%
GPT-5.1	59	57	53	51	39	51.8%
DeepSeek V4 Pro	61	58	50	45	39	50.7%
Claude Sonnet 4.6	59	55	50	42	39	48.8%
Z.AI GLM 5 Turbo	62	52	47	39	38	47.6%
GPT-5 Mini	55	54	48	45	32	47.0%
GPT-5.2	59	52	44	41	38	46.8%
Grok 4.3 (Reasoning)	60	55	52	36	31	46.8%
Claude Opus 4.6	56	51	48	38	33	45.0%
Z.AI GLM 5	63	55	47	34	25	44.9%
Claude Opus 4.8 (Reasoning, Low)	61	53	42	40	28	44.7%
Z.AI GLM 5.2 (Reasoning, High)	60	55	43	37	28	44.6%
Claude Haiku 4.5	62	60	50	27	20	43.9%
DeepSeek V4 Flash	50	47	45	41	36	43.7%
Qwen 3.5 9B	76	44	41	32	23	43.2%
Grok 4.5 (Reasoning, High)	61	51	44	33	26	42.8%
DeepSeek V4 Flash (Reasoning)	59	57	49	34	10	42.1%
Gemini 3.5 Flash (Reasoning, Minimal)	53	51	45	35	25	41.8%
Aion 3.0 Mini	65	53	42	27	18	41.1%
Writer: Palmyra X5	54	43	42	42	20	40.2%
MiniMax M2.7	46	43	39	36	32	39.3%
GPT-5.4 Nano	52	45	37	35	27	39.2%
Cydonia 24B V4.1	69	55	26	23	22	39.0%
Z.AI GLM 5.1	53	44	38	34	26	39.0%
DeepSeek V4 Pro (Reasoning)	57	45	39	32	22	38.9%
MoonshotAI: Kimi K2.6	67	54	26	24	24	38.8%
Grok 4.20 (Reasoning)	49	46	40	28	26	38.0%
Qwen 3.5 Flash	67	46	45	22	10	37.9%
DeepSeek V3.2	52	48	41	30	16	37.4%
GPT-5.4 Nano (Reasoning, Low)	48	40	35	34	27	37.1%
GPT-5.4 Nano (Reasoning)	58	42	37	32	16	37.0%
Z.AI GLM 4.7 Flash	54	50	41	21	12	35.6%
MiniMax M2.5	58	51	29	22	17	35.2%
Grok 4.5 (Reasoning, Low)	55	40	38	26	16	34.9%
Ministral 3 14B	49	43	36	26	21	34.9%
Claude Opus 4.5	50	43	43	24	12	34.5%
Gemini 3.5 Flash (Reasoning)	44	41	40	34	11	34.1%
Mistral Medium 3.1	42	41	37	27	23	34.0%
Claude Sonnet 4.5	39	37	36	35	20	33.5%
Claude Sonnet 5 (Reasoning, Low)	59	48	45	14	0	33.1%
Aion 3.0	52	43	40	15	15	33.0%
Z.AI GLM 4.5 Air	59	54	38	9	4	32.7%
Z.AI GLM 4.5	60	45	31	17	7	32.1%
GPT-5 Nano	40	36	31	29	24	32.1%
Qwen 3.5 Plus (2026-02-15)	49	48	34	28	0	31.9%
Claude Opus 4.6 (Reasoning)	55	46	40	19	0	31.9%
Qwen3 235B A22B Instruct 2507	51	46	27	20	16	31.9%
ByteDance Seed 1.6	58	41	26	17	16	31.8%
Grok 4.3	42	35	33	26	22	31.6%
Qwen 3.5 35B	45	42	40	24	0	30.2%
Qwen 3.5 27B	52	41	23	20	7	28.4%
Qwen 3.5 122B	50	33	28	16	7	26.7%
Xiaomi MIMO v2.5	48	31	28	20	0	25.3%
Xiaomi MIMO v2.5 Pro	50	32	25	19	0	25.2%
MoonshotAI: Kimi K2.5	43	35	31	6	3	23.5%
Z.AI GLM 4.7	48	41	14	11	0	22.9%
Claude Opus 4	45	25	24	20	0	22.9%
Qwen 3 32B	48	30	18	15	0	22.3%
Qwen3.6 Max Preview	32	26	24	19	6	21.6%
Qwen3.7 Max	46	28	23	9	0	21.4%
Mistral Small 4	38	22	20	16	8	20.9%
Aion 2.0	56	25	14	8	0	20.6%
Grok 4.20	54	18	14	8	6	20.0%
Claude Sonnet 4	42	28	16	14	0	19.9%
Ministral 8B	40	27	15	13	0	19.0%
Qwen 3.5 397B A17B	30	21	16	15	13	19.0%
Cohere Command R+ (Aug. 2024)	41	31	17	2	0	18.4%
Gemini 3 Flash (Preview, Reasoning)	30	27	24	9	0	17.9%
Hermes 3 70B	46	21	18	0	0	17.1%
DeepSeek V3 (2025-03-24)	31	29	16	5	0	16.2%
Qwen 3.6 35B	23	21	16	9	8	15.6%
Qwen 3.5 Plus (2026-04-20)	33	28	11	4	0	15.3%
Mistral Small 4 (Reasoning)	29	27	17	0	0	14.5%
Z.AI GLM 4.6	24	19	17	0	0	12.0%
Qwen 3.6 Flash	25	22	11	1	0	11.8%
Mistral Large 3	24	24	9	0	0	11.4%
GPT-4.1	31	14	8	0	0	10.7%
WizardLM 2 8x22b	22	14	7	5	5	10.6%
Gemini 3.1 Flash Lite (Reasoning)	17	14	14	7	0	10.3%
Nemotron 3 Super	27	13	8	0	0	9.5%
Gemini 3.1 Pro (Preview)	18	11	11	8	0	9.5%
Mistral Large 2	26	11	9	0	0	9.3%
Ministral 3 8B	32	13	1	0	0	9.2%
Gemini 2.5 Pro	16	15	14	0	0	9.0%
Hermes 3 405B	23	19	2	0	0	8.8%
Gemini 3 Flash (Preview)	22	15	5	0	0	8.5%
Gemma 4 31B	25	8	7	0	0	7.9%
DeepSeek-V2 Chat	21	11	4	0	0	7.3%
Qwen 3.6 27B	18	16	2	0	0	7.2%
Gemini 3.1 Flash Lite (Preview)	22	7	3	0	0	6.5%
Ministral 3B	19	10	0	0	0	5.8%
o4 Mini	17	8	3	1	0	5.6%
Ministral 3 3B	15	9	0	0	0	4.8%
DeepSeek V3.1	18	5	0	0	0	4.7%
Gemini 3.1 Flash Lite	18	3	0	0	0	4.1%
Gemma 3 27B	14	6	0	0	0	3.9%
Gemma 4 26B	7	6	0	0	0	2.5%
Mistral NeMO	12	0	0	0	0	2.4%
Gemini 2.5 Flash Lite (Reasoning)	11	0	0	0	0	2.2%
Qwen 2.5 72B	7	3	0	0	0	2.1%
DeepSeek V3 (2024-12-26)	6	2	0	0	0	1.6%
GPT-4.1 Mini	7	0	0	0	0	1.5%
Gemini 2.5 Flash Lite	5	0	0	0	0	1.0%
o4 Mini High	5	0	0	0	0	1.0%
Gemini 2.5 Flash	4	0	0	0	0	0.8%
Gemma 4 31B (Reasoning)	0	0	0	0	0	0.0%
Gemma 4 26B (Reasoning)	0	0	0	0	0	0.0%
GPT-OSS 120B	0	0	0	0	0	0.0%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.5 (Reasoning, Low)	80	73	69	67	65	70.5%
ByteDance Seed 2.0 Lite	84	76	69	66	57	70.3%
GPT-5.5	76	73	72	70	60	70.2%
Claude Opus 4.7 (Reasoning)	81	76	73	62	47	67.7%
GPT-5	77	67	66	63	59	66.3%
ByteDance Seed 2.0 Mini	79	71	67	56	50	64.8%
GPT-5.4 (Reasoning)	72	70	65	61	54	64.3%
Claude Opus 4.8 (Reasoning)	76	72	70	51	46	63.0%
GPT-5.4 Mini (Reasoning)	66	64	63	61	54	61.4%
GPT-5.5 (Reasoning)	70	63	59	59	55	61.0%
Claude Opus 4.7	72	71	58	57	42	60.1%
Claude Sonnet 5 (Reasoning)	71	59	58	56	53	59.6%
Claude Sonnet 5 (Reasoning, Low)	78	65	57	53	43	59.0%
GPT-5.4 (Reasoning, Low)	72	65	59	50	49	58.9%
MiniMax M3	63	63	60	54	53	58.6%
Claude Sonnet 5	66	66	57	57	45	58.4%
GPT-5.4	59	59	56	55	42	54.1%
GPT-5.4 Mini	59	55	54	53	48	53.6%
GPT-5.4 Mini (Reasoning, Low)	70	63	52	44	36	53.0%
ByteDance Seed 1.6 Flash	60	58	51	46	38	50.4%
Grok 4.5 (Reasoning, High)	60	57	50	42	41	50.2%
MiniMax M2.5	64	54	45	44	37	48.9%
GPT-5.1	59	55	52	46	32	48.9%
Claude Opus 4.6 (Reasoning)	65	50	48	42	39	48.6%
Claude Sonnet 4.6 (Reasoning)	55	54	48	43	43	48.5%
Grok 4.20	57	53	51	38	34	46.8%
Claude Sonnet 4.6	74	52	48	44	14	46.6%
Z.AI GLM 5.2 (Reasoning, High)	62	54	46	35	32	45.9%
Qwen 3.5 9B	98	58	33	29	9	45.3%
GPT-5.2	53	49	44	40	38	44.9%
Claude Opus 4.8 (Reasoning, Low)	55	49	45	38	37	44.8%
Grok 4.3 (Reasoning)	64	54	52	27	25	44.5%
Qwen 3.5 Flash	51	51	39	39	35	43.0%
GPT-5 Mini	58	49	48	34	24	42.4%
Claude Opus 4.6	59	54	40	30	29	42.4%
MiniMax M2.7	53	50	38	36	33	41.9%
Z.AI GLM 5	50	43	42	40	30	41.2%
Grok 4.5 (Reasoning, Low)	57	50	43	28	27	40.8%
Qwen 3.5 35B	45	42	42	40	29	39.7%
Z.AI GLM 5 Turbo	62	54	53	20	8	39.5%
DeepSeek V4 Pro (Reasoning)	45	43	38	35	32	38.7%
Z.AI GLM 4.7 Flash	53	40	39	35	24	38.0%
Qwen 3.5 Plus (2026-04-20)	66	53	36	31	4	38.0%
Qwen3.6 Max Preview	57	44	41	28	17	37.5%
MoonshotAI: Kimi K2.5	56	54	52	25	0	37.3%
GPT-5.4 Nano (Reasoning, Low)	48	43	38	31	27	37.2%
ByteDance Seed 1.6	56	43	41	31	12	36.6%
Claude Opus 4.5	55	40	38	32	18	36.5%
DeepSeek V4 Flash	57	38	32	28	26	36.4%
DeepSeek V4 Pro	57	36	36	31	17	35.3%
GPT-5.4 Nano	42	41	38	34	21	34.9%
Aion 3.0	45	40	39	32	17	34.8%
Mistral Small 4 (Reasoning)	55	47	28	24	16	33.8%
Gemini 3 Flash (Preview, Reasoning)	51	42	28	28	20	33.8%
Z.AI GLM 4.7	56	40	37	20	15	33.7%
Grok 4.20 (Reasoning)	50	46	38	29	4	33.5%
Z.AI GLM 5.1	53	52	41	20	0	33.4%
GPT-4.1	48	38	28	28	24	33.3%
GPT-5 Nano	47	42	30	24	22	33.2%
MoonshotAI: Kimi K2.6	46	44	36	20	17	32.7%
Mistral Small 4	57	40	34	24	5	31.9%
Qwen 3 32B	51	47	30	26	0	30.8%
GPT-5.4 Nano (Reasoning)	50	35	34	31	0	30.1%
Gemini 3 Flash (Preview)	38	37	34	23	18	30.1%
DeepSeek V4 Flash (Reasoning)	45	37	37	17	11	29.4%
Grok 4.3	52	36	32	16	9	29.2%
Qwen 3.5 122B	40	27	27	27	23	28.9%
Gemini 3.5 Flash (Reasoning, Minimal)	39	37	36	25	1	27.8%
Claude Haiku 4.5	42	40	25	19	6	26.3%
Claude Opus 4	62	22	18	17	9	25.4%
Qwen3 235B A22B Instruct 2507	38	23	23	22	20	25.1%
Qwen 3.5 27B	39	31	19	18	17	25.0%
Qwen 3.5 397B A17B	52	32	29	13	0	24.9%
Qwen 3.6 35B	36	34	27	21	6	24.8%
Aion 3.0 Mini	45	33	31	14	0	24.5%
Xiaomi MIMO v2.5 Pro	52	31	24	12	4	24.5%
Mistral Large 3	43	40	23	13	0	23.6%
Claude Sonnet 4.5	49	27	22	21	0	23.6%
Aion 2.0	50	36	19	2	0	21.4%
Gemini 3.5 Flash (Reasoning)	29	28	25	23	0	20.7%
Writer: Palmyra X5	40	34	14	10	0	19.7%
Qwen3.7 Max	47	25	13	13	0	19.7%
Mistral Small 3.2 24B	97	0	0	0	0	19.5%
Mistral Medium 3.1	28	21	20	19	0	17.6%
Qwen 3.6 Flash	23	20	20	19	5	17.4%
Qwen 3.5 Plus (2026-02-15)	33	19	19	9	0	16.0%
Xiaomi MIMO v2.5	30	28	11	6	0	14.8%
Z.AI GLM 4.5 Air	38	23	9	0	0	14.1%
Gemini 3.1 Pro (Preview)	42	25	0	0	0	13.5%
WizardLM 2 8x22b	38	15	8	7	0	13.5%
Gemini 3.1 Flash Lite	31	18	9	8	0	13.1%
Qwen 3.6 27B	37	24	4	0	0	13.0%
Z.AI GLM 4.5	27	25	0	0	0	10.3%
Cydonia 24B V4.1	28	14	6	4	0	10.2%
DeepSeek V3.2	38	10	2	0	0	9.9%
Claude Sonnet 4	26	16	4	0	0	9.4%
Gemma 3 27B	25	15	6	0	0	9.2%
DeepSeek V3.1	41	2	0	0	0	8.6%
DeepSeek V3 (2025-03-24)	16	11	8	4	0	7.9%
Ministral 3 8B	34	2	0	0	0	7.0%
Gemma 4 31B	21	11	3	0	0	7.0%
Z.AI GLM 4.6	24	6	5	0	0	6.9%
Hermes 3 70B	18	12	3	0	0	6.7%
Gemma 4 26B	17	10	6	0	0	6.6%
Gemini 3.1 Flash Lite (Reasoning)	17	11	4	0	0	6.3%
Arcee AI: Trinity Mini	29	0	0	0	0	5.8%
Gemini 3.1 Flash Lite (Preview)	16	10	0	0	0	5.2%
DeepSeek-V2 Chat	24	0	0	0	0	4.8%
Mistral Large 2	18	4	0	0	0	4.4%
Ministral 8B	7	6	5	0	0	3.7%
Gemma 4 26B (Reasoning)	18	0	0	0	0	3.6%
GPT-OSS 120B	17	0	0	0	0	3.5%
Hermes 3 405B	17	0	0	0	0	3.3%
Ministral 3B	16	0	0	0	0	3.1%
DeepSeek V3 (2024-12-26)	9	3	0	0	0	2.5%
Ministral 3 14B	7	0	0	0	0	1.4%
Nemotron 3 Super	6	0	0	0	0	1.3%
Gemma 3 12B	5	0	0	0	0	0.9%
Gemini 2.5 Pro	3	2	0	0	0	0.9%
o4 Mini High	4	0	0	0	0	0.9%
o4 Mini	4	0	0	0	0	0.8%
Gemma 3 4B	3	0	0	0	0	0.7%
Gemma 4 31B (Reasoning)	3	0	0	0	0	0.6%
Qwen 2.5 72B	3	0	0	0	0	0.6%
Cohere Command R+ (Aug. 2024)	2	0	0	0	0	0.5%
Gemini 2.5 Flash (Reasoning)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Inception Mercury 2	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Nemotron 3 Nano	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 5	96	95	95	91	85	92.5%
Claude Opus 4.7	95	92	91	89	86	90.5%
Claude Opus 4.7 (Reasoning)	96	91	91	87	86	90.0%
GPT-5.5 (Reasoning, Low)	92	90	89	89	87	89.5%
Claude Opus 4.8 (Reasoning, Low)	95	91	91	86	78	88.4%
GPT-5.5 (Reasoning)	90	90	87	86	85	87.7%
GPT-5.4 (Reasoning, Low)	88	88	86	86	86	86.6%
Claude Sonnet 4.6 (Reasoning)	96	90	87	79	78	85.9%
Claude Opus 4.8 (Reasoning)	92	92	91	83	71	85.8%
Claude Sonnet 5 (Reasoning)	92	86	82	82	82	84.9%
GPT-5.5	89	87	86	85	77	84.6%
GPT-5.4 Mini (Reasoning)	90	87	84	81	80	84.5%
GPT-5	86	86	84	83	81	83.9%
GPT-5.4	86	85	82	82	80	83.0%
GPT-5.4 (Reasoning)	88	84	84	81	76	82.8%
Claude Sonnet 4.6	91	86	81	76	76	82.2%
GPT-5.4 Mini (Reasoning, Low)	89	81	81	80	78	81.8%
Claude Sonnet 5 (Reasoning, Low)	86	81	81	81	77	81.2%
GPT-5.1	86	82	80	79	74	80.2%
ByteDance Seed 2.0 Lite	89	82	80	79	69	79.6%
Claude Opus 4.6	89	84	82	73	65	78.4%
MiniMax M3	91	83	76	71	71	78.3%
ByteDance Seed 1.6 Flash	93	82	74	70	70	78.1%
GPT-5.4 Mini	85	84	77	73	71	78.0%
Claude Opus 4.6 (Reasoning)	81	80	80	74	73	77.6%
ByteDance Seed 2.0 Mini	88	85	82	74	58	77.6%
Grok 4.3 (Reasoning)	83	79	77	76	68	76.3%
Z.AI GLM 5 Turbo	87	85	79	72	57	76.2%
GPT-5 Mini	83	81	80	74	63	76.2%
Aion 3.0	89	77	76	72	65	75.9%
Aion 3.0 Mini	83	83	77	73	53	73.9%
GPT-5.2	79	77	72	71	70	73.7%
Claude Haiku 4.5	79	73	72	72	72	73.3%
Z.AI GLM 5.2 (Reasoning, High)	90	72	69	67	64	72.4%
Z.AI GLM 5	79	77	73	66	61	71.2%
DeepSeek V4 Flash	82	78	70	69	57	71.2%
Claude Opus 4.5	74	72	70	70	65	70.4%
GPT-5.4 Nano (Reasoning)	76	75	68	66	63	69.6%
MiniMax M2.7	79	78	74	65	53	69.6%
Qwen 3.5 397B A17B	73	72	71	70	61	69.5%
MiniMax M2.5	81	74	68	65	55	68.7%
DeepSeek V4 Pro (Reasoning)	79	71	68	65	58	68.2%
Grok 4.3	79	76	67	67	51	68.0%
GPT-5.4 Nano (Reasoning, Low)	76	72	72	61	56	67.6%
Claude Sonnet 4.5	77	69	69	64	58	67.6%
GPT-5.4 Nano	74	70	68	66	60	67.5%
Grok 4.5 (Reasoning, Low)	77	75	68	65	52	67.3%
Xiaomi MIMO v2.5	70	68	68	63	55	64.7%
Writer: Palmyra X5	78	69	63	60	53	64.6%
Z.AI GLM 5.1	79	69	64	58	50	64.2%
Qwen3.6 Max Preview	70	67	65	58	54	62.9%
MoonshotAI: Kimi K2.6	78	67	62	59	49	62.8%
Grok 4.5 (Reasoning, High)	66	65	62	60	58	62.4%
Qwen 3.5 122B	79	65	59	58	50	62.3%
Claude Opus 4	69	65	63	60	54	62.2%
Qwen 3.5 9B	82	78	57	52	37	61.4%
Qwen 3.5 35B	78	69	64	53	42	61.3%
DeepSeek V4 Pro	78	77	58	54	39	61.2%
Qwen 3.6 27B	68	62	59	58	55	60.1%
DeepSeek V4 Flash (Reasoning)	72	66	58	55	49	60.0%
Qwen3 235B A22B Instruct 2507	65	62	60	56	55	59.9%
GPT-5 Nano	61	60	58	58	55	58.4%
Qwen 3.5 Flash	66	65	61	51	46	57.5%
ByteDance Seed 1.6	71	65	64	54	30	56.6%
Mistral Small 4 (Reasoning)	68	63	52	49	48	56.1%
Gemini 3.5 Flash (Reasoning)	62	56	55	53	50	55.1%
DeepSeek V3.2	72	61	59	47	31	54.0%
Gemini 3.5 Flash (Reasoning, Minimal)	68	61	58	42	38	53.3%
Qwen 3.5 27B	80	58	52	38	37	53.1%
Qwen 3.5 Plus (2026-02-15)	67	61	58	43	36	52.8%
Gemma 4 31B (Reasoning)	65	62	52	44	39	52.3%
Gemini 3 Flash (Preview, Reasoning)	77	50	49	45	41	52.3%
Xiaomi MIMO v2.5 Pro	62	55	51	47	47	52.3%
WizardLM 2 8x22b	63	62	49	43	43	52.0%
GPT-4.1	64	60	55	42	32	50.7%
MoonshotAI: Kimi K2.5	67	51	51	47	34	50.1%
Z.AI GLM 4.6	64	59	53	45	29	49.9%
Gemini 3.1 Pro (Preview)	72	61	58	32	26	49.8%
Aion 2.0	62	53	46	43	42	49.1%
Gemini 3 Flash (Preview)	61	54	53	38	38	48.9%
Mistral Large 3	74	66	57	32	14	48.5%
Grok 4.20	58	53	47	47	35	48.1%
Hermes 3 70B	67	49	47	43	33	47.5%
Grok 4.20 (Reasoning)	58	53	46	40	39	47.4%
Mistral Small 4	70	43	43	42	39	47.1%
DeepSeek V3 (2025-03-24)	64	52	45	37	32	45.9%
Z.AI GLM 4.7 Flash	54	50	48	45	31	45.7%
Ministral 3 14B	72	59	46	42	9	45.5%
Z.AI GLM 4.7	69	63	37	31	26	45.3%
Qwen 3.5 Plus (2026-04-20)	54	54	42	36	34	44.0%
Ministral 8B	66	61	61	20	10	43.6%
Mistral Large 2	69	44	38	37	25	42.6%
Z.AI GLM 4.5 Air	54	51	43	37	27	42.4%
Z.AI GLM 4.5	53	47	38	37	35	42.1%
Mistral Medium 3.1	55	52	43	36	22	41.6%
Qwen 3 32B	61	60	39	36	8	40.8%
DeepSeek V3.1	66	45	43	25	23	40.4%
Qwen 3.6 Flash	57	47	33	32	32	40.3%
Claude Sonnet 4	51	46	42	35	27	40.3%
Qwen 3.6 35B	65	44	38	37	5	37.9%
Qwen3.7 Max	49	49	39	31	21	37.8%
o4 Mini High	58	48	42	40	0	37.6%
Cohere Command R+ (Aug. 2024)	70	39	34	29	0	34.5%
Cydonia 24B V4.1	44	43	35	19	16	31.3%
Gemma 3 27B	43	35	31	23	17	29.9%
Hermes 3 405B	53	42	24	20	7	29.5%
Nemotron 3 Super	60	41	32	14	0	29.3%
GPT-4.1 Mini	49	42	28	27	0	29.2%
Gemini 2.5 Pro	40	39	37	16	14	29.1%
Gemini 3.1 Flash Lite (Preview)	48	36	24	22	13	28.6%
Gemini 2.5 Flash (Reasoning)	39	37	31	22	15	28.6%
Mistral NeMO	58	43	43	0	0	28.6%
Gemini 3.1 Flash Lite (Reasoning)	44	34	30	28	5	28.4%
DeepSeek V3 (2024-12-26)	64	32	31	8	0	27.0%
Gemini 3.1 Flash Lite	33	32	25	25	19	26.7%
o4 Mini	34	28	27	24	12	24.9%
Gemma 4 31B	45	45	16	8	7	24.3%
Gemma 4 26B	46	35	16	14	10	24.2%
Gemini 2.5 Flash Lite (Reasoning)	36	32	22	16	11	23.6%
Ministral 3B	48	35	33	0	0	23.2%
Ministral 3 8B	42	29	21	14	10	23.2%
GPT-4.1 Nano	34	26	23	19	9	22.3%
DeepSeek-V2 Chat	41	26	22	21	0	22.0%
Qwen 2.5 72B	46	37	18	0	0	20.1%
Gemma 4 26B (Reasoning)	32	22	19	15	10	19.7%
Gemma 3 12B	30	24	16	4	0	14.8%
GPT-4o, Aug. 6th (temp=1)	31	15	12	6	0	12.9%
GPT-OSS 120B	27	27	10	0	0	12.8%
Gemini 2.5 Flash Lite	22	22	0	0	0	8.6%
Llama 3.1 70B	37	0	0	0	0	7.5%
Ministral 3 3B	28	7	0	0	0	6.9%
Gemma 3 4B	17	16	2	0	0	6.9%
Gemini 2.5 Flash	17	16	0	0	0	6.7%
Mistral Small 3.2 24B	29	2	0	0	0	6.1%
Nemotron 3 Nano	12	8	6	0	0	5.2%
Arcee AI: Trinity Mini	14	6	1	1	0	4.3%
GPT-4o, Aug. 6th (temp=0)	10	4	3	0	0	3.4%
Inception Mercury 2	9	4	3	0	0	3.3%
GPT-4o Mini (temp=1)	4	0	0	0	0	0.8%
GPT-4o Mini (temp=0)	1	0	0	0	0	0.1%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 5 (Reasoning)	100	95	90	89	80	90.8%
Claude Opus 4.7 (Reasoning)	96	95	90	86	86	90.7%
Claude Sonnet 5	95	95	91	85	80	89.2%
Claude Opus 4.7	92	92	91	86	82	88.7%
Claude Sonnet 4.6	96	95	84	81	81	87.4%
GPT-5.4	93	91	85	85	76	86.0%
GPT-5.4 Mini (Reasoning)	89	88	88	82	82	85.7%
GPT-5.4 (Reasoning)	93	92	83	83	77	85.7%
GPT-5.4 (Reasoning, Low)	92	89	87	80	74	84.5%
Claude Sonnet 4.6 (Reasoning)	87	87	87	83	77	84.2%
GPT-5.5	89	88	83	80	77	83.4%
GPT-5.5 (Reasoning, Low)	90	89	87	80	70	83.0%
GPT-5.4 Mini (Reasoning, Low)	91	89	84	79	72	82.8%
Z.AI GLM 5 Turbo	92	85	75	74	74	80.1%
GPT-5.5 (Reasoning)	86	85	79	78	72	79.9%
GPT-5.4 Mini	83	82	81	78	76	79.8%
GPT-5	86	83	79	73	72	78.8%
Qwen 3.5 9B	100	84	81	75	49	77.7%
GPT-5 Mini	86	83	80	69	68	77.3%
Claude Opus 4.8 (Reasoning, Low)	87	80	77	74	68	77.1%
Claude Opus 4.6	83	79	78	78	67	77.0%
ByteDance Seed 1.6 Flash	88	81	79	73	62	76.7%
ByteDance Seed 2.0 Lite	80	78	77	76	69	76.1%
MiniMax M3	92	84	82	81	41	76.0%
GPT-5.4 Nano (Reasoning)	87	81	76	68	67	75.8%
GPT-5.4 Nano (Reasoning, Low)	81	78	76	74	69	75.7%
Claude Opus 4.6 (Reasoning)	90	83	76	72	56	75.6%
Claude Sonnet 5 (Reasoning, Low)	95	80	76	67	58	75.3%
Claude Opus 4.8 (Reasoning)	87	86	69	69	63	74.9%
GPT-5.1	81	81	73	69	68	74.3%
Claude Sonnet 4.5	83	75	74	69	68	73.9%
Grok 4.5 (Reasoning, High)	82	75	73	69	67	73.3%
DeepSeek V4 Pro	88	74	73	68	61	72.9%
MiniMax M2.5	82	79	78	69	54	72.3%
Aion 3.0 Mini	84	74	70	70	64	72.2%
GPT-5.2	83	79	71	64	62	72.1%
Xiaomi MIMO v2.5	80	74	71	70	60	71.1%
Aion 3.0	82	78	68	64	60	70.5%
ByteDance Seed 1.6	77	74	74	70	57	70.2%
Z.AI GLM 5.2 (Reasoning, High)	86	71	66	65	60	69.3%
Grok 4.5 (Reasoning, Low)	88	84	61	59	49	68.2%
Z.AI GLM 5.1	84	70	69	63	53	67.7%
Claude Haiku 4.5	86	69	69	61	50	67.2%
GPT-5.4 Nano	73	67	66	64	64	66.8%
ByteDance Seed 2.0 Mini	72	72	66	63	58	66.3%
DeepSeek V4 Pro (Reasoning)	74	74	72	63	47	66.1%
Claude Opus 4	83	81	63	59	42	65.9%
Claude Opus 4.5	71	70	64	63	62	65.9%
MiniMax M2.7	84	77	72	49	48	65.8%
DeepSeek V4 Flash (Reasoning)	80	73	58	56	54	64.2%
MoonshotAI: Kimi K2.6	69	66	64	58	50	61.4%
Mistral Small 4 (Reasoning)	77	70	66	58	37	61.3%
Qwen 3.5 Flash	78	74	58	49	44	60.7%
Z.AI GLM 4.7 Flash	71	67	56	55	53	60.3%
Z.AI GLM 5	66	65	63	63	43	60.1%
Qwen 3.5 122B	79	79	59	44	39	59.9%
Claude Sonnet 4	76	65	64	63	27	59.0%
DeepSeek V4 Flash	73	64	62	51	41	58.2%
Qwen 3.5 27B	69	64	61	50	44	57.6%
GPT-5 Nano	59	58	57	56	55	56.9%
WizardLM 2 8x22b	63	59	56	55	51	56.8%
DeepSeek V3.2	68	60	59	58	39	56.7%
Grok 4.3 (Reasoning)	65	65	57	52	43	56.3%
MoonshotAI: Kimi K2.5	71	64	57	50	38	56.2%
Grok 4.3	78	73	47	46	36	56.0%
Aion 2.0	64	59	59	50	47	56.0%
Z.AI GLM 4.5 Air	74	55	52	50	47	55.6%
Mistral Medium 3.1	71	68	64	39	35	55.4%
Grok 4.20	67	57	52	49	40	53.1%
Z.AI GLM 4.7	55	53	50	49	46	50.6%
Mistral Large 3	70	62	62	47	11	50.5%
Qwen 3.5 35B	69	61	53	35	34	50.3%
GPT-4.1	65	63	41	41	39	49.9%
Z.AI GLM 4.5	61	49	48	46	44	49.9%
Grok 4.20 (Reasoning)	72	53	48	42	33	49.7%
Xiaomi MIMO v2.5 Pro	70	54	52	41	30	49.4%
Qwen 3.5 397B A17B	71	56	47	35	34	48.9%
Qwen3.6 Max Preview	60	50	45	44	44	48.6%
Qwen 3.5 Plus (2026-04-20)	61	60	56	47	15	47.8%
Ministral 3 8B	60	52	51	45	29	47.4%
Gemini 3 Flash (Preview)	59	52	48	40	38	47.3%
Writer: Palmyra X5	64	58	38	37	36	46.5%
Gemini 3.5 Flash (Reasoning, Minimal)	58	52	43	42	36	46.3%
Ministral 8B	59	49	42	40	38	45.8%
Qwen 3.6 Flash	63	61	43	32	26	45.0%
Qwen 3 32B	68	51	40	35	27	44.3%
Qwen3.7 Max	61	51	48	38	22	44.0%
Qwen 3.6 35B	69	58	38	36	18	43.6%
Gemini 3.5 Flash (Reasoning)	51	51	43	38	34	43.4%
Cydonia 24B V4.1	56	45	44	42	29	43.3%
GPT-4.1 Mini	58	47	41	34	33	42.8%
Gemini 3 Flash (Preview, Reasoning)	54	53	51	30	24	42.6%
Z.AI GLM 4.6	56	45	41	41	29	42.4%
Mistral Large 2	66	66	42	36	0	41.7%
Qwen 3.6 27B	67	52	48	36	2	41.0%
Cohere Command R+ (Aug. 2024)	63	48	41	40	4	39.5%
Llama 3.1 70B	92	55	17	16	15	39.1%
Gemini 3.1 Pro (Preview)	75	35	33	25	22	37.9%
Qwen3 235B A22B Instruct 2507	46	42	37	35	29	37.7%
Arcee AI: Trinity Mini	64	53	42	26	0	36.9%
DeepSeek V3.1	56	54	45	26	0	36.5%
Nemotron 3 Super	57	46	45	21	11	36.4%
o4 Mini	49	42	38	31	20	36.2%
Gemini 2.5 Flash	49	47	43	40	1	35.9%
Mistral Small 4	49	45	35	23	21	34.6%
Hermes 3 70B	57	37	35	33	11	34.4%
Gemma 3 27B	43	41	38	31	18	34.3%
Gemini 2.5 Flash (Reasoning)	47	36	32	24	20	31.9%
GPT-4.1 Nano	49	41	33	18	18	31.7%
Gemini 3.1 Flash Lite (Reasoning)	60	47	26	23	0	31.1%
DeepSeek-V2 Chat	46	37	34	24	10	30.3%
Gemma 4 31B	64	33	24	23	7	30.2%
DeepSeek V3 (2025-03-24)	46	38	34	18	16	30.2%
Gemini 2.5 Pro	42	40	29	21	19	30.2%
Qwen 3.5 Plus (2026-02-15)	46	39	36	19	10	29.7%
Gemini 3.1 Flash Lite (Preview)	44	28	27	27	22	29.7%
Gemini 3.1 Flash Lite	48	38	36	25	0	29.5%
Hermes 3 405B	46	35	28	20	15	28.6%
Gemma 4 31B (Reasoning)	43	32	26	25	13	27.8%
DeepSeek V3 (2024-12-26)	37	35	32	27	0	26.1%
Ministral 3B	41	31	31	25	0	25.6%
Gemma 4 26B	38	31	26	25	5	25.0%
Mistral Small 3.2 24B	98	27	0	0	0	25.0%
Gemma 3 12B	50	35	22	14	0	24.0%
GPT-OSS 120B	48	31	22	17	0	23.6%
Ministral 3 14B	41	39	16	15	3	23.0%
Inception Mercury 2	29	28	26	16	15	22.8%
GPT-4o, Aug. 6th (temp=0)	51	30	18	14	0	22.6%
o4 Mini High	51	22	16	8	0	19.5%
Gemini 2.5 Flash Lite (Reasoning)	41	33	5	0	0	15.9%
Gemini 2.5 Flash Lite	40	10	10	8	5	14.7%
Gemma 4 26B (Reasoning)	29	14	9	5	3	11.9%
Qwen 2.5 72B	24	15	10	9	1	11.8%
Nemotron 3 Nano	31	28	0	0	0	11.8%
Ministral 3 3B	27	22	8	0	0	11.5%
GPT-4o Mini (temp=1)	27	12	0	0	0	7.7%
Gemma 3 4B	13	6	5	4	0	5.5%
Mistral NeMO	20	7	0	0	0	5.4%
GPT-4o Mini (temp=0)	7	0	0	0	0	1.5%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 5 (Reasoning)	93	90	90	87	85	89.1%
ByteDance Seed 2.0 Mini	93	90	89	87	85	88.6%
Claude Opus 4.7 (Reasoning)	95	91	87	82	80	86.9%
ByteDance Seed 2.0 Lite	88	87	84	83	81	84.9%
Claude Opus 4.8 (Reasoning, Low)	96	91	82	81	72	84.5%
Claude Sonnet 5 (Reasoning, Low)	95	86	85	80	75	84.0%
GPT-5.4	89	83	83	83	80	83.6%
Claude Sonnet 4.6	100	90	79	74	74	83.5%
Qwen 3.5 9B	88	87	85	85	71	83.1%
Claude Opus 4.7	95	95	77	75	72	82.9%
MiniMax M3	88	87	79	78	75	81.5%
GPT-5.5 (Reasoning)	88	85	84	81	68	81.2%
Claude Sonnet 5	90	84	80	76	73	80.5%
Z.AI GLM 5.2 (Reasoning, High)	85	82	81	78	73	79.9%
GPT-5	84	83	78	77	77	79.8%
GPT-5 Mini	84	82	78	78	69	78.2%
GPT-5.4 Mini (Reasoning, Low)	82	77	76	76	75	77.2%
Claude Opus 4.8 (Reasoning)	91	81	77	73	63	77.0%
GPT-5.5 (Reasoning, Low)	81	79	79	78	68	76.9%
GPT-5.4 (Reasoning, Low)	86	80	79	74	66	76.8%
GPT-5.4 (Reasoning)	89	81	78	68	66	76.4%
GPT-5.5	87	79	75	72	67	76.0%
Claude Sonnet 4.6 (Reasoning)	86	80	77	74	62	75.9%
ByteDance Seed 1.6 Flash	84	79	77	72	67	75.6%
GPT-5.4 Mini (Reasoning)	81	78	77	77	64	75.4%
Aion 3.0 Mini	79	78	77	73	70	75.3%
DeepSeek V4 Pro	85	81	77	66	65	74.8%
MiniMax M2.5	81	76	76	75	58	73.1%
ByteDance Seed 1.6	89	75	70	69	63	73.1%
Grok 4.3 (Reasoning)	84	72	72	70	67	73.0%
GPT-5.4 Nano (Reasoning, Low)	76	74	72	71	71	72.8%
DeepSeek V4 Flash	79	77	77	66	65	72.6%
Claude Opus 4.6	82	82	71	68	60	72.4%
GPT-5.4 Mini	77	77	74	69	61	71.8%
Grok 4.5 (Reasoning, Low)	79	75	71	70	62	71.1%
Z.AI GLM 5 Turbo	85	82	64	61	59	70.4%
GPT-5.4 Nano (Reasoning)	77	72	68	65	64	69.2%
Claude Opus 4.6 (Reasoning)	84	68	68	64	61	69.1%
GPT-5.4 Nano	74	68	67	67	62	67.6%
Qwen3.7 Max	92	68	65	57	52	66.9%
GPT-5.1	75	70	68	66	54	66.6%
Qwen 3.5 397B A17B	77	70	67	59	55	65.5%
Aion 3.0	84	67	64	61	52	65.5%
Claude Opus 4.5	80	72	67	62	46	65.3%
Qwen3.6 Max Preview	83	74	63	58	48	65.1%
GPT-5.2	69	68	65	63	60	64.9%
Claude Haiku 4.5	78	78	66	57	44	64.5%
Writer: Palmyra X5	74	72	72	63	41	64.5%
Qwen3 235B A22B Instruct 2507	72	69	67	63	51	64.2%
MoonshotAI: Kimi K2.6	84	71	57	55	54	64.0%
DeepSeek V4 Flash (Reasoning)	79	71	65	52	49	63.1%
Xiaomi MIMO v2.5 Pro	78	72	70	51	44	63.0%
Xiaomi MIMO v2.5	75	72	57	55	53	62.5%
Z.AI GLM 4.7	68	68	68	57	51	62.3%
Claude Sonnet 4.5	84	72	56	53	45	62.2%
Grok 4.5 (Reasoning, High)	68	68	59	58	58	62.1%
MiniMax M2.7	75	73	65	64	24	60.5%
Z.AI GLM 5	71	60	59	58	50	59.7%
Z.AI GLM 5.1	75	64	64	58	34	58.9%
Claude Opus 4	71	65	56	56	40	57.6%
Qwen 3.6 35B	64	63	56	55	47	57.2%
DeepSeek V4 Pro (Reasoning)	67	64	62	54	38	57.0%
Mistral Large 3	72	55	54	53	48	56.4%
Qwen 3.5 122B	69	69	62	49	32	56.2%
Mistral Small 4	61	59	58	56	46	55.8%
MoonshotAI: Kimi K2.5	76	62	49	47	40	54.9%
WizardLM 2 8x22b	68	62	55	53	35	54.7%
Grok 4.20	65	57	52	50	49	54.6%
Qwen 3.5 Flash	65	65	55	49	37	54.1%
GPT-5 Nano	63	57	51	51	49	54.0%
Ministral 3 14B	78	76	47	36	32	53.8%
Z.AI GLM 4.5	70	66	58	42	33	53.6%
Grok 4.3	62	59	54	50	42	53.3%
Aion 2.0	62	58	58	48	40	53.1%
Gemini 3 Flash (Preview)	65	59	52	45	41	52.2%
Mistral Large 2	81	60	50	39	31	52.2%
GPT-4.1	72	52	46	44	42	51.2%
Qwen 3.6 27B	60	59	55	48	33	51.1%
Qwen 3 32B	61	56	54	51	32	50.9%
DeepSeek V3.2	51	50	49	47	41	47.7%
Qwen 3.5 35B	68	60	42	38	28	47.2%
DeepSeek V3 (2025-03-24)	68	58	45	35	29	47.0%
Mistral Medium 3.1	72	57	43	39	23	46.8%
DeepSeek V3.1	64	61	51	29	28	46.6%
Z.AI GLM 4.6	60	58	47	34	34	46.6%
Qwen 3.5 Plus (2026-02-15)	73	53	41	37	25	45.8%
Qwen 3.5 27B	67	47	44	40	31	45.8%
Qwen 3.5 Plus (2026-04-20)	61	56	56	28	24	45.2%
Mistral Small 4 (Reasoning)	58	51	48	39	28	44.9%
DeepSeek V3 (2024-12-26)	63	52	45	38	25	44.7%
Grok 4.20 (Reasoning)	60	51	38	37	34	43.7%
Z.AI GLM 4.7 Flash	51	48	39	38	32	41.7%
Qwen 3.6 Flash	57	55	52	39	3	41.1%
Gemini 3.5 Flash (Reasoning)	51	50	42	37	19	39.9%
Cydonia 24B V4.1	63	48	46	40	0	39.2%
Claude Sonnet 4	60	54	33	27	17	38.2%
Gemini 3.5 Flash (Reasoning, Minimal)	44	42	41	36	24	37.3%
Gemini 3 Flash (Preview, Reasoning)	48	37	35	32	28	35.9%
Ministral 3 8B	58	49	46	19	8	35.7%
Hermes 3 70B	64	44	38	30	0	35.2%
Hermes 3 405B	66	37	36	25	8	34.6%
o4 Mini	51	45	25	25	25	34.4%
Z.AI GLM 4.5 Air	44	42	32	31	18	33.5%
Nemotron 3 Super	47	37	34	29	15	32.4%
Cohere Command R+ (Aug. 2024)	64	58	28	10	0	31.8%
Ministral 8B	48	43	38	21	9	31.8%
Gemma 4 31B	59	44	19	18	17	31.5%
o4 Mini High	49	43	25	22	8	29.4%
Gemini 2.5 Pro	43	39	38	17	9	29.3%
DeepSeek-V2 Chat	40	34	33	25	12	28.8%
Gemma 4 31B (Reasoning)	36	35	33	20	18	28.5%
Gemma 4 26B	36	35	35	25	3	26.7%
Gemma 4 26B (Reasoning)	42	38	31	21	0	26.5%
GPT-OSS 120B	41	28	27	20	14	26.2%
Gemini 3.1 Pro (Preview)	58	43	20	5	0	25.1%
Gemini 3.1 Flash Lite (Preview)	40	38	27	16	0	24.0%
Ministral 3 3B	46	40	32	0	0	23.6%
Gemma 3 27B	32	28	25	25	1	22.1%
Inception Mercury 2	42	34	25	0	0	20.3%
Mistral NeMO	51	23	19	6	1	20.0%
Gemini 3.1 Flash Lite	49	29	9	7	0	18.5%
Arcee AI: Trinity Mini	47	22	17	4	2	18.1%
Qwen 2.5 72B	47	21	14	6	0	17.7%
Gemini 3.1 Flash Lite (Reasoning)	30	25	23	9	0	17.5%
Gemma 3 4B	44	28	13	0	0	17.1%
Llama 3.1 70B	77	0	0	0	0	15.4%
Gemma 3 12B	27	21	19	6	1	14.9%
Gemini 2.5 Flash	29	22	14	1	0	13.0%
Gemini 2.5 Flash Lite (Reasoning)	32	19	0	0	0	10.0%
GPT-4o, Aug. 6th (temp=1)	24	22	0	0	0	9.2%
GPT-4.1 Nano	23	10	9	0	0	8.4%
GPT-4.1 Mini	26	10	5	0	0	8.2%
Gemini 2.5 Flash (Reasoning)	24	8	1	0	0	6.6%
Mistral Small 3.2 24B	22	10	0	0	0	6.3%
Gemini 2.5 Flash Lite	13	2	0	0	0	3.0%
Nemotron 3 Nano	9	4	2	0	0	2.9%
Ministral 3B	11	0	0	0	0	2.2%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	93	88	86	82	82	86.1%
GPT-5.5 (Reasoning)	84	84	83	80	79	82.0%
Claude Sonnet 4.6	90	86	80	76	76	81.7%
Claude Sonnet 4.6 (Reasoning)	95	86	80	73	71	81.0%
GPT-5.4	89	82	81	80	71	80.4%
Claude Sonnet 5	86	86	85	80	65	80.3%
Claude Opus 4.7	90	82	81	76	71	80.0%
ByteDance Seed 2.0 Lite	90	81	77	76	73	79.4%
GPT-5.5	83	82	81	77	74	79.4%
GPT-5.4 (Reasoning, Low)	85	84	82	82	63	79.3%
GPT-5.4 Mini (Reasoning)	90	77	77	76	74	78.9%
GPT-5	85	84	79	75	69	78.5%
Claude Sonnet 5 (Reasoning)	91	86	81	74	54	77.1%
Claude Sonnet 5 (Reasoning, Low)	87	80	79	72	67	76.8%
GPT-5.5 (Reasoning, Low)	82	81	77	73	68	76.2%
GPT-5 Mini	86	79	78	68	67	75.6%
Claude Opus 4.8 (Reasoning)	81	81	77	71	67	75.5%
Claude Opus 4.7 (Reasoning)	80	78	77	73	67	74.9%
GPT-5.4 Mini	81	80	74	70	68	74.6%
ByteDance Seed 2.0 Mini	85	84	78	64	55	73.3%
GPT-5.2	77	74	74	73	66	72.9%
Claude Opus 4.8 (Reasoning, Low)	81	77	76	65	62	72.0%
GPT-5.4 Nano (Reasoning, Low)	82	79	73	70	55	71.6%
MiniMax M3	76	76	75	67	63	71.3%
ByteDance Seed 1.6 Flash	86	77	72	64	56	71.2%
Claude Opus 4.5	76	76	70	69	65	71.2%
GPT-5.4 Nano (Reasoning)	77	75	72	72	55	70.2%
GPT-5.4 Mini (Reasoning, Low)	86	82	70	59	53	70.2%
Claude Opus 4.6	82	77	72	67	53	70.1%
MiniMax M2.5	84	72	69	67	53	69.0%
Grok 4.5 (Reasoning, Low)	80	80	68	66	44	67.6%
Qwen 3.5 9B	94	66	64	57	54	66.9%
Claude Opus 4	82	69	66	59	54	66.0%
MiniMax M2.7	85	76	71	59	36	65.4%
Qwen 3.5 35B	82	75	66	60	44	65.4%
MoonshotAI: Kimi K2.6	72	69	63	63	60	65.3%
Aion 3.0	87	64	64	62	45	64.5%
Z.AI GLM 5 Turbo	91	69	65	53	44	64.3%
GPT-5.4 Nano	78	66	61	61	53	63.8%
GPT-5.1	74	69	69	60	47	63.7%
Z.AI GLM 4.7	75	66	66	57	54	63.7%
Claude Opus 4.6 (Reasoning)	74	70	68	56	47	63.1%
Grok 4.5 (Reasoning, High)	89	75	63	48	40	63.0%
Z.AI GLM 5.2 (Reasoning, High)	70	65	65	56	52	61.7%
Claude Sonnet 4.5	69	66	62	55	53	61.0%
WizardLM 2 8x22b	89	59	59	52	37	59.1%
Gemini 3.5 Flash (Reasoning, Minimal)	70	64	56	54	52	58.9%
ByteDance Seed 1.6	62	61	58	57	56	58.7%
Z.AI GLM 5	75	63	54	50	37	56.0%
Aion 2.0	64	54	53	52	49	54.5%
DeepSeek V4 Pro	68	66	48	46	43	54.2%
Writer: Palmyra X5	67	61	54	53	37	54.2%
Qwen 3.5 397B A17B	64	63	62	47	33	53.8%
DeepSeek V4 Flash (Reasoning)	71	54	50	49	44	53.7%
DeepSeek V3.2	66	65	57	47	33	53.6%
Claude Sonnet 4	58	57	56	51	45	53.5%
Grok 4.20 (Reasoning)	68	66	63	35	32	52.9%
Z.AI GLM 4.7 Flash	78	65	49	38	32	52.4%
DeepSeek V4 Flash	69	65	56	42	31	52.4%
Xiaomi MIMO v2.5 Pro	86	71	41	36	26	52.2%
Z.AI GLM 5.1	69	65	46	46	33	51.7%
Qwen 3.5 27B	64	61	51	43	39	51.5%
Z.AI GLM 4.5	73	50	49	44	41	51.3%
Gemini 3.5 Flash (Reasoning)	67	62	50	40	34	50.5%
Z.AI GLM 4.6	71	65	46	45	21	49.5%
Aion 3.0 Mini	79	73	52	31	12	49.4%
GPT-5 Nano	51	50	48	47	45	48.0%
Qwen3.7 Max	57	55	54	47	26	47.9%
Claude Haiku 4.5	66	55	46	39	32	47.5%
Grok 4.3 (Reasoning)	71	52	47	40	25	47.1%
Qwen 3.5 122B	54	54	43	40	39	46.0%
Qwen 3.6 Flash	51	51	48	42	31	44.6%
DeepSeek V3 (2025-03-24)	63	50	45	43	21	44.3%
Cydonia 24B V4.1	61	59	56	24	20	43.9%
Mistral Large 2	61	47	39	36	35	43.5%
Qwen 3.5 Flash	59	53	42	38	22	42.8%
Gemini 3.1 Pro (Preview)	63	51	37	32	31	42.7%
Qwen 3.6 35B	66	44	41	37	25	42.7%
MoonshotAI: Kimi K2.5	85	46	37	36	10	42.6%
Qwen 3.5 Plus (2026-02-15)	59	58	44	27	25	42.5%
Gemini 3 Flash (Preview)	52	49	46	35	30	42.4%
Qwen3 235B A22B Instruct 2507	67	55	38	29	18	41.4%
DeepSeek V3.1	54	42	39	36	33	40.8%
Grok 4.20	57	49	44	27	25	40.6%
Mistral Medium 3.1	54	49	34	33	28	39.6%
Grok 4.3	54	46	40	32	27	39.5%
Ministral 8B	58	55	37	22	18	38.2%
DeepSeek V4 Pro (Reasoning)	64	63	26	20	19	38.2%
Mistral Small 4 (Reasoning)	60	47	42	20	13	36.4%
Z.AI GLM 4.5 Air	57	53	44	21	6	36.3%
Cohere Command R+ (Aug. 2024)	66	43	36	33	0	35.7%
Gemma 4 31B	42	42	35	30	27	35.2%
Qwen 3.5 Plus (2026-04-20)	55	45	39	28	4	34.2%
Xiaomi MIMO v2.5	73	44	28	19	3	33.5%
Gemini 3 Flash (Preview, Reasoning)	35	34	33	33	31	33.1%
Qwen3.6 Max Preview	57	44	24	24	10	31.8%
Qwen 3 32B	70	38	33	16	0	31.4%
GPT-4.1	39	38	31	24	23	31.0%
Hermes 3 70B	44	42	27	26	13	30.3%
Ministral 3 8B	85	42	22	1	0	29.9%
Qwen 3.6 27B	48	47	23	20	10	29.4%
Gemma 4 31B (Reasoning)	62	43	32	4	0	28.3%
Ministral 3 14B	42	38	26	14	10	26.0%
Gemma 3 27B	50	35	31	11	3	25.8%
Gemini 3.1 Flash Lite (Reasoning)	42	42	20	16	2	24.4%
Mistral Small 4	42	22	22	19	13	23.7%
DeepSeek-V2 Chat	49	28	21	14	5	23.3%
Mistral Large 3	39	35	14	10	8	21.3%
Gemma 3 12B	34	24	23	14	11	21.0%
o4 Mini High	38	32	20	12	0	20.4%
Ministral 3 3B	49	28	15	8	0	20.2%
Gemini 2.5 Pro	33	21	20	13	12	20.0%
Gemini 3.1 Flash Lite	46	29	19	4	0	19.9%
DeepSeek V3 (2024-12-26)	40	32	15	12	0	19.7%
Gemma 4 26B	37	28	21	1	0	17.7%
GPT-4.1 Mini	40	32	15	0	0	17.5%
Gemma 4 26B (Reasoning)	36	22	16	11	0	17.0%
Ministral 3B	33	26	19	4	0	16.4%
Gemma 3 4B	41	11	9	8	4	14.9%
GPT-4.1 Nano	37	19	18	0	0	14.8%
Nemotron 3 Super	27	17	15	10	4	14.7%
Arcee AI: Trinity Mini	32	16	14	9	0	14.1%
Mistral Small 3.2 24B	30	28	0	0	0	11.6%
Hermes 3 405B	31	14	11	1	0	11.4%
Qwen 2.5 72B	30	14	8	0	0	10.3%
Llama 3.1 70B	32	13	5	0	0	10.0%
Gemini 3.1 Flash Lite (Preview)	19	13	9	7	0	9.5%
o4 Mini	25	18	0	0	0	8.6%
Nemotron 3 Nano	20	0	0	0	0	4.1%
Gemini 2.5 Flash Lite	12	3	2	0	0	3.3%
GPT-4o Mini (temp=1)	16	0	0	0	0	3.3%
GPT-OSS 120B	11	2	1	0	0	2.9%
Gemini 2.5 Flash (Reasoning)	13	0	0	0	0	2.6%
Inception Mercury 2	5	5	0	0	0	2.1%
Mistral NeMO	9	0	0	0	0	1.8%
GPT-4o Mini (temp=0)	5	0	0	0	0	1.0%
Gemini 2.5 Flash	4	0	0	0	0	0.8%
Gemini 2.5 Flash Lite (Reasoning)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%

AI-ism word frequency

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets