AI-ism word frequency

Test: Bad Writing Habits

Avg. Score

31.8%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	ByteDance Seed 1.6 Flash	69.1%	$0.0013	27.3s	51%
2	GPT-5 Mini	66.0%	$0.0100	57.4s	49%
3	Claude Sonnet 4.6	70.1%	$0.031	39.3s	48%
4	Minimax M2.5	60.6%	$0.0034	1.3m	44%
5	Claude Haiku 4.5	57.6%	$0.011	21.6s	38%
6	GPT-5	74.3%	$0.065	2.8m	62%
7	Z.AI GLM 5	54.9%	$0.0084	1.2m	39%
8	GPT-5.2	61.7%	$0.056	1.5m	48%
9	GPT-5.1	63.5%	$0.054	1.8m	48%
10	Claude Opus 4.6	63.8%	$0.078	1.2m	47%
11	Claude Sonnet 4.5	54.8%	$0.035	38.1s	35%
12	Writer: Palmyra X5	46.9%	$0.011	22.0s	32%
13	Gemini 3 Flash (Preview)	41.4%	$0.0078	19.6s	30%
14	Claude Opus 4.5	58.1%	$0.070	53.4s	41%
15	Z.AI GLM 4.7 Flash	46.0%	$0.0017	1.2m	34%
16	GPT-5 Nano	46.7%	$0.0042	1.4m	36%
17	Z.AI GLM 4.7	48.0%	$0.010	1.4m	34%
18	Z.AI GLM 4.5	43.2%	$0.0051	42.1s	29%
19	Claude 3.7 Sonnet	49.3%	$0.042	46.7s	33%
20	Mistral Small Creative	36.1%	$0.0007	9.1s	24%
21	Mistral Medium 3.1	39.2%	$0.0048	36.5s	28%
22	Qwen 3.5 Plus (2026-02-15)	38.9%	$0.0060	31.5s	26%
23	ByteDance Seed 1.6	54.9%	$0.013	2.5m	35%
24	Rocinante 12B	43.4%	$0.0014	38.4s	20%
25	Ministral 3 14B	33.4%	$0.0007	11.7s	20%
26	Llama 3.1 8B	38.9%	$0.0003	1.3m	26%
27	GPT-4.1	38.3%	$0.018	44.7s	25%
28	DeepSeek V3 (2025-03-24)	36.2%	$0.0014	39.4s	21%
29	Mistral Large 2	37.0%	$0.013	29.4s	21%
30	Gemini 3 Pro (Preview)	45.3%	$0.055	54.4s	31%
31	Mistral Large 3	35.0%	$0.0033	30.3s	20%
32	Mistral Large	36.6%	$0.014	30.9s	21%
33	Arcee AI: Trinity Large (Preview)	35.8%	$0.0000	43.6s	19%
34	Claude Sonnet 4	39.3%	$0.032	43.7s	24%
35	DeepSeek V3.2	39.5%	$0.0014	1.9m	26%
36	Ministral 8B	27.2%	$0.0004	10.4s	15%
37	Ministral 3 8B	28.4%	$0.0008	19.6s	14%
38	Z.AI GLM 4.6	31.6%	$0.0065	51.5s	19%
39	Qwen 3.5 397B A17B	46.9%	$0.014	3.0m	31%
40	Hermes 3 70B	30.1%	$0.0010	1.2m	18%
41	Hermes 3 405B	32.1%	$0.0032	53.2s	14%
42	MoonshotAI: Kimi K2.5	46.7%	$0.019	3.2m	31%
43	Claude 3.5 Sonnet	34.1%	$0.048	35.5s	19%
44	WizardLM 2 8x22b	33.9%	$0.0026	1.8m	17%
45	DeepSeek V3.1	30.5%	$0.0020	1.8m	19%
46	Gemma 3 27B	21.2%	$0.0006	52.6s	16%
47	DeepSeek V3 (2024-12-26)	23.9%	$0.0021	54.6s	13%
48	Gemini 2.5 Pro	26.8%	$0.036	36.2s	17%
49	DeepSeek-V2 Chat	22.0%	$0.0021	53.3s	12%
50	Ministral 3B	17.4%	$0.0001	8.1s	7%
51	Grok 4 Fast	15.5%	$0.0017	24.1s	11%
52	o4 Mini	17.8%	$0.015	25.7s	12%
53	Grok 4.1 Fast	17.5%	$0.0018	37.8s	10%
54	Ministral 3 3B	13.6%	$0.0005	11.1s	5%
55	Arcee AI: Trinity Mini	13.1%	$0.0003	9.2s	1%
56	Gemini 2.5 Flash	12.8%	$0.0052	10.6s	2%
57	Gemma 3 12B	13.1%	$0.0004	41.3s	5%
58	Mistral NeMO	10.7%	$0.0005	10.1s	1%
59	o4 Mini High	16.3%	$0.025	47.2s	10%
60	GPT-4.1 Mini	11.8%	$0.0027	19.0s	0%
61	Claude 3.5 Haiku	10.1%	$0.0035	10.8s	0%
62	Claude Opus 4	54.8%	$0.209	1.4m	37%
63	Llama 3.1 70B	13.4%	$0.0015	29.4s	0%
64	Cohere Command R+ (Aug. 2024)	20.1%	$0.020	52.5s	4%
65	Gemini 2.5 Flash Lite	8.1%	$0.0009	9.5s	0%
66	Claude 3 Haiku	9.7%	$0.0025	14.9s	0%
67	GPT-4.1 Nano	8.1%	$0.0007	13.3s	0%
68	Gemma 3 4B	8.3%	$0.0002	20.0s	0%
69	Stealth: Aurora Alpha	4.5%	$0.0000	9.8s	0%
70	Gemini 3.1 Pro (Preview)	35.0%	$0.107	1.8m	23%
71	Qwen 2.5 72B	8.0%	$0.0010	36.7s	0%
72	Grok 4	19.4%	$0.048	1.7m	14%
73	Llama 3.1 Nemotron 70B	5.0%	$0.0038	31.7s	0%
74	GPT-4o Mini (temp=1)	2.3%	$0.0012	34.8s	0%
75	GPT-4o, May 13th (temp=0)	8.9%	$0.035	14.1s	0%
76	GPT-4o Mini (temp=0)	0.8%	$0.0012	34.8s	0%
77	GPT-4o, May 13th (temp=1)	7.5%	$0.033	14.4s	0%
78	GPT-4o, Aug. 6th (temp=1)	2.9%	$0.018	24.4s	0%
79	GPT-4o, Aug. 6th (temp=0)	3.1%	$0.023	22.7s	0%
80	Mistral Small 3.2 24B	12.0%	$0.0069	5.7m	0%
31.76%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
ByteDance Seed 1.6 Flash	82	77	69	55	48	66.2%
GPT-5	75	63	63	56	56	62.5%
GPT-5 Mini	76	60	59	51	48	58.8%
GPT-5.2	68	66	59	51	45	57.5%
Claude Sonnet 4.6	66	65	51	50	29	52.0%
GPT-5.1	68	60	43	42	37	49.9%
Claude Opus 4.6	64	49	46	45	44	49.8%
Claude Opus 4.5	64	54	54	40	16	45.4%
Claude Opus 4	61	51	47	35	32	45.1%
ByteDance Seed 1.6	63	50	37	36	31	43.5%
Minimax M2.5	67	62	34	28	25	43.2%
Claude 3.7 Sonnet	53	39	39	39	35	41.0%
Z.AI GLM 4.7	53	46	40	33	30	40.5%
Z.AI GLM 4.7 Flash	60	54	47	41	0	40.2%
Writer: Palmyra X5	63	58	55	21	0	39.4%
Z.AI GLM 4.5	54	54	48	21	19	39.3%
Qwen 3.5 397B A17B	56	52	43	25	20	39.0%
Qwen 3.5 Plus (2026-02-15)	69	46	32	24	10	36.3%
GPT-5 Nano	41	41	36	34	27	35.9%
DeepSeek V3 (2025-03-24)	55	44	39	33	0	34.0%
Claude Haiku 4.5	45	35	32	32	25	33.7%
Rocinante 12B	59	58	39	11	0	33.5%
Z.AI GLM 5	43	35	27	25	18	29.7%
Gemini 3 Flash (Preview)	45	39	28	20	15	29.6%
MoonshotAI: Kimi K2.5	62	25	24	22	14	29.4%
Hermes 3 70B	58	34	28	25	0	28.9%
Claude Sonnet 4.5	40	35	30	24	13	28.5%
Gemini 3 Pro (Preview)	52	31	31	13	11	27.7%
Llama 3.1 8B	54	43	38	0	0	27.1%
Mistral Medium 3.1	35	32	27	18	12	25.0%
WizardLM 2 8x22b	54	38	32	0	0	24.6%
Gemini 3.1 Pro (Preview)	44	40	27	11	0	24.4%
Mistral Large 3	34	33	18	17	13	23.1%
Ministral 3 14B	49	43	21	1	0	22.8%
DeepSeek V3.2	48	25	21	19	0	22.5%
Mistral Large	41	23	21	14	8	21.6%
Hermes 3 405B	39	28	24	14	0	21.1%
DeepSeek V3 (2024-12-26)	41	39	25	0	0	21.1%
GPT-4.1	31	21	20	15	14	20.1%
Mistral Large 2	41	22	19	16	0	19.7%
Gemini 2.5 Pro	33	28	14	14	0	17.8%
Llama 3.1 70B	88	0	0	0	0	17.6%
Ministral 3B	67	20	0	0	0	17.5%
Claude 3.5 Sonnet	27	24	16	16	2	17.0%
Mistral Small Creative	24	23	14	14	0	15.2%
Arcee AI: Trinity Large (Preview)	40	31	2	0	0	14.4%
Ministral 3 8B	28	27	16	0	0	14.1%
Z.AI GLM 4.6	20	15	13	6	0	10.6%
Cohere Command R+ (Aug. 2024)	51	0	0	0	0	10.2%
Claude Sonnet 4	37	5	3	0	0	9.1%
DeepSeek-V2 Chat	30	15	0	0	0	8.9%
Grok 4.1 Fast	18	14	11	0	0	8.6%
DeepSeek V3.1	19	12	7	0	0	7.6%
Arcee AI: Trinity Mini	37	0	0	0	0	7.5%
Ministral 8B	29	6	0	0	0	6.9%
Mistral NeMO	11	10	0	0	0	4.3%
Gemma 3 27B	13	8	0	0	0	4.2%
Grok 4 Fast	11	1	0	0	0	2.4%
Ministral 3 3B	12	0	0	0	0	2.4%
Gemini 2.5 Flash	6	0	0	0	0	1.1%
GPT-4o, May 13th (temp=1)	4	0	0	0	0	0.8%
Gemma 3 4B	4	0	0	0	0	0.8%
Mistral Small 3.2 24B	3	0	0	0	0	0.7%
Grok 4	0	0	0	0	0	0.1%
o4 Mini High	0	0	0	0	0	0.0%
o4 Mini	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	74	73	70	69	65	70.2%
ByteDance Seed 1.6 Flash	75	63	58	54	48	59.6%
Rocinante 12B	100	75	53	36	14	55.7%
Claude Opus 4.6	70	67	49	46	41	54.7%
GPT-5.1	73	59	50	49	36	53.7%
GPT-5.2	65	51	50	50	19	47.2%
GPT-5 Mini	63	61	50	35	25	46.6%
Minimax M2.5	67	57	54	27	27	46.5%
Claude Opus 4	65	60	38	37	30	45.7%
Claude Sonnet 4.5	75	53	47	35	18	45.4%
Qwen 3.5 397B A17B	58	53	44	40	31	45.3%
Claude Haiku 4.5	69	49	41	36	31	45.2%
ByteDance Seed 1.6	46	45	41	36	36	40.7%
Z.AI GLM 4.7	56	52	45	36	13	40.2%
Arcee AI: Trinity Large (Preview)	72	46	32	23	22	39.1%
Claude Opus 4.5	48	44	40	32	31	38.9%
Z.AI GLM 4.7 Flash	51	51	40	33	18	38.6%
GPT-5 Nano	50	40	38	35	30	38.5%
Claude Sonnet 4.6	60	37	36	33	26	38.3%
Z.AI GLM 5	64	40	38	25	21	37.5%
MoonshotAI: Kimi K2.5	61	38	32	29	24	36.8%
Qwen 3.5 Plus (2026-02-15)	52	37	33	31	16	33.8%
Gemini 3 Flash (Preview)	51	38	30	29	20	33.7%
Gemini 3 Pro (Preview)	48	44	32	30	14	33.6%
Hermes 3 405B	60	54	39	13	0	33.3%
DeepSeek V3 (2025-03-24)	67	53	39	7	0	33.1%
Claude 3.7 Sonnet	41	40	35	18	15	29.8%
WizardLM 2 8x22b	46	42	28	15	0	26.2%
Writer: Palmyra X5	39	30	26	20	15	25.8%
DeepSeek V3 (2024-12-26)	67	52	0	0	0	24.0%
Mistral Small Creative	50	29	25	10	4	23.6%
DeepSeek V3.2	36	34	27	17	0	22.9%
GPT-4.1	34	27	19	19	15	22.7%
Mistral Medium 3.1	32	31	24	16	9	22.4%
Gemini 3.1 Pro (Preview)	47	41	20	3	0	22.2%
Claude Sonnet 4	46	38	26	0	0	22.0%
Claude 3.5 Sonnet	40	35	21	0	0	19.4%
Llama 3.1 8B	52	22	18	0	0	18.2%
Hermes 3 70B	62	20	0	0	0	16.2%
Mistral Large 2	50	19	9	0	0	15.4%
DeepSeek-V2 Chat	75	0	0	0	0	15.0%
Mistral Large	42	17	8	5	0	14.6%
Z.AI GLM 4.5	34	16	16	2	0	13.6%
Llama 3.1 70B	62	0	0	0	0	12.4%
Ministral 3 8B	27	19	9	0	0	10.8%
Gemini 2.5 Pro	27	27	0	0	0	10.8%
Grok 4.1 Fast	28	14	10	0	0	10.4%
Gemma 3 27B	28	12	0	0	0	8.1%
Gemma 3 12B	26	0	0	0	0	5.1%
Mistral Small 3.2 24B	17	9	0	0	0	5.1%
DeepSeek V3.1	17	7	0	0	0	4.8%
Ministral 3 3B	18	6	0	0	0	4.8%
Ministral 3 14B	13	10	0	0	0	4.6%
Z.AI GLM 4.6	23	0	0	0	0	4.5%
Stealth: Aurora Alpha	9	0	0	0	0	1.8%
Mistral Large 3	5	1	0	0	0	1.3%
Cohere Command R+ (Aug. 2024)	6	0	0	0	0	1.1%
Gemini 2.5 Flash	5	0	0	0	0	1.0%
Gemma 3 4B	2	0	0	0	0	0.4%
o4 Mini High	0	0	0	0	0	0.0%
o4 Mini	0	0	0	0	0	0.0%
Grok 4	0	0	0	0	0	0.0%
Grok 4 Fast	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=1)	0	0	0	0	0	0.0%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
Qwen 2.5 72B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Ministral 8B	0	0	0	0	0	0.0%
Ministral 3B	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	92	88	87	84	79	86.2%
ByteDance Seed 1.6 Flash	94	91	89	76	72	84.2%
GPT-5.1	82	82	82	72	69	77.3%
Claude Haiku 4.5	87	80	77	74	66	77.0%
GPT-5.2	82	78	78	69	66	74.6%
GPT-5 Mini	88	76	71	69	60	73.0%
Claude Sonnet 4.6	85	77	73	71	57	72.5%
Claude Opus 4.5	85	77	67	67	63	71.7%
Claude Sonnet 4.5	89	79	70	59	56	70.5%
Claude Opus 4.6	78	77	72	61	61	69.9%
Hermes 3 405B	93	77	68	54	52	68.8%
Claude Opus 4	81	79	68	63	52	68.6%
Z.AI GLM 4.7	81	73	73	64	51	68.5%
Qwen 3.5 397B A17B	80	71	69	59	53	66.5%
Z.AI GLM 5	72	70	68	63	59	66.2%
Gemini 3 Pro (Preview)	75	70	67	59	56	65.4%
Minimax M2.5	80	70	69	67	28	62.7%
Z.AI GLM 4.7 Flash	79	70	65	54	40	61.8%
ByteDance Seed 1.6	78	69	66	48	45	61.1%
Writer: Palmyra X5	74	72	62	58	39	60.8%
DeepSeek V3.2	67	63	62	56	40	57.8%
MoonshotAI: Kimi K2.5	68	60	58	56	46	57.4%
Claude 3.7 Sonnet	76	75	52	42	40	56.7%
Gemini 3 Flash (Preview)	67	63	58	57	37	56.3%
Z.AI GLM 4.5	70	56	55	52	46	55.7%
Qwen 3.5 Plus (2026-02-15)	69	57	54	51	38	53.9%
GPT-4.1	79	66	53	39	30	53.5%
WizardLM 2 8x22b	66	58	53	47	43	53.4%
Claude Sonnet 4	67	65	53	52	31	53.4%
DeepSeek V3 (2025-03-24)	71	66	55	54	19	53.1%
Mistral Large 3	66	58	58	41	39	52.1%
Mistral Large 2	69	68	53	43	21	50.9%
Claude 3.5 Sonnet	70	64	60	47	12	50.6%
Ministral 8B	78	51	50	36	33	49.4%
Gemini 3.1 Pro (Preview)	60	54	44	42	35	47.1%
Rocinante 12B	100	57	40	35	2	46.7%
GPT-5 Nano	50	49	45	45	44	46.5%
Mistral Medium 3.1	69	59	43	31	30	46.3%
Ministral 3 8B	59	58	55	54	0	45.2%
DeepSeek V3.1	56	52	48	35	28	43.9%
Llama 3.1 8B	57	56	51	30	23	43.4%
Mistral Large	59	43	42	37	34	42.9%
Gemini 2.5 Pro	65	48	46	32	17	41.6%
DeepSeek V3 (2024-12-26)	78	60	37	20	10	41.0%
Hermes 3 70B	85	45	37	36	0	40.6%
Gemma 3 27B	45	39	38	38	36	39.2%
Ministral 3 14B	59	38	34	34	25	38.0%
Arcee AI: Trinity Large (Preview)	67	34	31	30	26	37.7%
Grok 4	55	43	39	29	21	37.2%
Z.AI GLM 4.6	72	31	30	30	18	36.2%
o4 Mini	45	35	34	33	22	33.7%
Mistral Small Creative	50	50	40	21	1	32.2%
Grok 4.1 Fast	55	42	40	21	0	31.5%
Grok 4 Fast	33	32	24	22	15	25.2%
Arcee AI: Trinity Mini	59	45	20	0	0	24.9%
DeepSeek-V2 Chat	35	33	25	20	0	22.6%
Cohere Command R+ (Aug. 2024)	59	44	0	0	0	20.8%
Gemma 3 12B	37	22	21	19	4	20.6%
GPT-4.1 Mini	53	21	17	1	0	18.4%
Llama 3.1 70B	71	21	0	0	0	18.3%
Gemini 2.5 Flash Lite	29	26	18	12	0	17.0%
Ministral 3B	52	31	0	0	0	16.6%
Gemini 2.5 Flash	44	33	5	0	0	16.5%
Mistral NeMO	56	11	9	4	0	15.8%
GPT-4o, May 13th (temp=1)	33	17	15	8	0	14.6%
Claude 3.5 Haiku	40	32	0	0	0	14.3%
Mistral Small 3.2 24B	62	0	0	0	0	12.5%
Claude 3 Haiku	52	5	0	0	0	11.5%
o4 Mini High	39	10	8	0	0	11.3%
Ministral 3 3B	38	10	0	0	0	9.6%
Gemma 3 4B	38	0	0	0	0	7.7%
Qwen 2.5 72B	15	0	0	0	0	2.9%
GPT-4o, Aug. 6th (temp=1)	8	4	0	0	0	2.4%
GPT-4o, Aug. 6th (temp=0)	12	0	0	0	0	2.3%
GPT-4.1 Nano	9	1	0	0	0	2.0%
GPT-4o Mini (temp=1)	8	2	0	0	0	2.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	95	91	86	86	67	84.9%
GPT-5	83	82	76	72	72	77.1%
ByteDance Seed 1.6 Flash	92	86	79	52	51	71.8%
Claude Haiku 4.5	80	77	73	69	55	70.9%
GPT-5.2	78	74	71	65	65	70.6%
Claude Opus 4.6	82	79	73	62	56	70.6%
GPT-5 Mini	72	71	70	70	64	69.4%
GPT-5.1	75	74	68	63	62	68.3%
Claude Opus 4	72	71	71	65	58	67.3%
Minimax M2.5	87	77	68	53	52	67.2%
Z.AI GLM 5	71	70	67	65	60	66.5%
Claude Sonnet 4.5	78	71	70	65	44	65.6%
Claude 3.7 Sonnet	81	79	65	52	50	65.3%
Claude Opus 4.5	85	84	59	50	44	64.6%
Mistral Large 2	69	64	64	61	57	63.0%
ByteDance Seed 1.6	76	71	67	48	43	61.1%
WizardLM 2 8x22b	73	66	56	55	50	60.0%
Hermes 3 405B	92	61	60	54	30	59.1%
Writer: Palmyra X5	72	63	63	54	42	59.0%
Claude Sonnet 4	72	70	58	47	47	58.5%
Mistral Large	71	68	58	56	35	57.6%
Rocinante 12B	73	65	57	52	38	57.0%
Z.AI GLM 4.7	64	57	54	47	47	53.9%
Z.AI GLM 4.5	82	53	50	49	29	52.4%
Arcee AI: Trinity Large (Preview)	87	55	47	41	32	52.4%
MoonshotAI: Kimi K2.5	60	59	50	44	37	50.0%
Gemini 3 Pro (Preview)	64	54	46	45	40	49.6%
Cohere Command R+ (Aug. 2024)	62	58	54	41	29	48.7%
Z.AI GLM 4.6	75	50	49	40	29	48.6%
GPT-5 Nano	53	52	44	44	43	47.3%
Mistral Medium 3.1	57	51	51	45	33	47.2%
Z.AI GLM 4.7 Flash	55	51	48	43	39	47.1%
Mistral Large 3	64	64	49	29	26	46.5%
DeepSeek V3.1	56	51	48	44	33	46.5%
Gemini 3 Flash (Preview)	65	49	44	42	29	46.0%
Mistral Small Creative	52	44	44	41	39	44.2%
Gemma 3 27B	60	47	46	35	28	43.2%
Ministral 3 14B	83	65	27	27	12	42.9%
Qwen 3.5 397B A17B	60	45	42	37	27	42.5%
Mistral Small 3.2 24B	97	77	36	0	0	41.9%
DeepSeek V3.2	67	50	41	23	19	40.1%
Llama 3.1 8B	65	55	39	32	0	38.3%
GPT-4.1	53	41	38	38	22	38.1%
Gemini 3.1 Pro (Preview)	56	39	37	27	23	36.3%
Qwen 3.5 Plus (2026-02-15)	46	39	33	29	26	34.6%
Gemini 2.5 Pro	45	39	36	36	17	34.5%
Gemma 3 12B	50	44	29	23	21	33.3%
Gemini 2.5 Flash	45	31	30	29	28	32.5%
Hermes 3 70B	89	40	34	0	0	32.4%
Grok 4	53	31	31	18	17	29.8%
Ministral 3B	79	37	19	11	0	29.3%
GPT-4o, May 13th (temp=1)	49	45	24	16	8	28.6%
DeepSeek-V2 Chat	47	31	27	19	13	27.4%
Ministral 8B	43	41	30	16	5	26.8%
Ministral 3 8B	69	33	28	0	0	25.9%
Claude 3 Haiku	55	32	27	7	1	24.3%
o4 Mini High	36	25	23	18	17	23.9%
DeepSeek V3 (2024-12-26)	30	29	26	17	15	23.7%
o4 Mini	40	33	21	20	0	22.8%
Grok 4.1 Fast	35	27	23	23	0	21.6%
DeepSeek V3 (2025-03-24)	50	28	15	13	0	21.2%
Ministral 3 3B	33	24	23	18	0	19.7%
Arcee AI: Trinity Mini	47	26	23	0	0	19.4%
Gemini 2.5 Flash Lite	51	37	0	0	0	17.7%
GPT-4.1 Mini	42	26	12	0	0	16.2%
Grok 4 Fast	23	23	19	7	0	14.4%
GPT-4.1 Nano	38	17	16	0	0	14.1%
Mistral NeMO	33	19	17	0	0	13.9%
Claude 3.5 Sonnet	36	17	14	1	0	13.6%
GPT-4o, May 13th (temp=0)	26	24	8	5	0	12.6%
Gemma 3 4B	21	15	10	10	2	11.8%
Llama 3.1 70B	32	15	0	0	0	9.4%
GPT-4o Mini (temp=0)	26	19	0	0	0	9.0%
Qwen 2.5 72B	30	3	0	0	0	6.4%
GPT-4o, Aug. 6th (temp=1)	17	4	3	0	0	5.0%
GPT-4o Mini (temp=1)	15	0	0	0	0	3.0%
Stealth: Aurora Alpha	7	5	2	0	0	2.9%
GPT-4o, Aug. 6th (temp=0)	10	2	0	0	0	2.5%
Llama 3.1 Nemotron 70B	10	2	0	0	0	2.4%
Claude 3.5 Haiku	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	88	80	76	65	65	74.9%
Claude Opus 4.6	84	82	74	62	58	71.9%
ByteDance Seed 1.6 Flash	79	78	75	71	56	71.7%
Claude Haiku 4.5	86	83	80	64	45	71.5%
GPT-5	81	73	68	64	63	69.8%
Hermes 3 405B	95	94	57	55	48	69.7%
GPT-5.1	79	77	65	64	59	68.7%
ByteDance Seed 1.6	91	77	63	56	55	68.4%
Claude Opus 4.5	77	74	68	62	54	67.2%
GPT-5 Mini	75	70	69	62	58	66.9%
WizardLM 2 8x22b	79	74	69	60	49	66.1%
Minimax M2.5	70	69	68	65	56	65.5%
Z.AI GLM 5	79	65	62	61	59	65.2%
Mistral Large 2	71	69	68	66	50	64.9%
Claude Opus 4	75	74	70	51	51	64.5%
Qwen 3.5 397B A17B	77	67	65	55	54	63.8%
Claude Sonnet 4.5	74	73	71	49	30	59.4%
Writer: Palmyra X5	74	73	67	58	25	59.4%
GPT-5.2	66	64	57	55	51	58.7%
Gemini 3 Pro (Preview)	66	63	61	50	49	57.8%
Arcee AI: Trinity Large (Preview)	82	75	53	47	29	57.2%
MoonshotAI: Kimi K2.5	76	66	54	49	34	55.7%
Z.AI GLM 4.7	59	58	57	53	42	54.0%
Z.AI GLM 4.7 Flash	70	62	58	47	30	53.4%
Gemini 3 Flash (Preview)	61	60	51	50	41	52.6%
Claude Sonnet 4	69	62	56	47	28	52.3%
Qwen 3.5 Plus (2026-02-15)	59	57	51	48	46	52.1%
Mistral Small Creative	68	63	57	39	24	50.1%
Gemini 2.5 Pro	62	54	52	48	32	49.7%
Gemini 3.1 Pro (Preview)	65	50	49	42	36	48.6%
Hermes 3 70B	67	48	41	41	40	47.2%
Rocinante 12B	86	85	40	22	0	46.5%
Z.AI GLM 4.5	58	56	41	37	36	45.8%
DeepSeek V3.1	70	59	46	31	22	45.7%
Claude 3.7 Sonnet	60	49	48	40	29	45.4%
Ministral 3 14B	75	45	40	33	31	44.8%
Llama 3.1 8B	67	62	56	30	4	44.0%
DeepSeek V3.2	57	55	40	39	28	43.8%
GPT-5 Nano	70	46	38	38	25	43.2%
Ministral 3B	75	57	37	26	21	43.2%
DeepSeek V3 (2025-03-24)	70	64	41	41	0	43.1%
Mistral Large 3	75	39	37	34	27	42.3%
Mistral Large	69	58	45	28	4	40.7%
GPT-4.1	46	43	38	37	33	39.3%
Mistral Medium 3.1	45	41	37	32	28	36.8%
Z.AI GLM 4.6	47	42	39	34	20	36.2%
Claude 3.5 Sonnet	62	50	32	17	0	32.2%
DeepSeek V3 (2024-12-26)	49	37	27	27	20	31.9%
Ministral 8B	60	48	32	9	7	31.0%
Grok 4.1 Fast	48	40	32	14	12	29.1%
Arcee AI: Trinity Mini	76	30	25	9	0	28.1%
Grok 4	40	40	25	19	13	27.5%
Mistral Small 3.2 24B	48	44	21	20	0	26.6%
Mistral NeMO	51	35	32	5	2	25.0%
Grok 4 Fast	48	37	21	16	0	24.3%
Ministral 3 3B	53	45	13	0	0	22.2%
Gemma 3 12B	61	45	3	0	0	21.9%
Gemma 3 27B	45	26	26	4	3	20.8%
Ministral 3 8B	50	28	21	2	0	20.3%
Cohere Command R+ (Aug. 2024)	73	22	5	0	0	20.1%
DeepSeek-V2 Chat	66	10	9	7	0	18.5%
o4 Mini High	32	24	23	0	0	15.7%
GPT-4.1 Nano	43	21	8	2	0	14.7%
Claude 3.5 Haiku	52	13	7	0	0	14.4%
Gemini 2.5 Flash	36	24	9	0	0	13.8%
o4 Mini	38	24	1	0	0	12.6%
GPT-4o, May 13th (temp=1)	23	16	8	3	0	9.8%
Claude 3 Haiku	22	13	8	6	0	9.8%
GPT-4.1 Mini	27	8	7	1	0	8.6%
GPT-4o, Aug. 6th (temp=0)	38	0	0	0	0	7.6%
GPT-4o, May 13th (temp=0)	38	0	0	0	0	7.5%
Llama 3.1 70B	35	0	0	0	0	7.1%
GPT-4o Mini (temp=1)	13	0	0	0	0	2.5%
Gemini 2.5 Flash Lite	12	0	0	0	0	2.4%
Llama 3.1 Nemotron 70B	10	0	0	0	0	2.0%
Stealth: Aurora Alpha	6	0	0	0	0	1.2%
GPT-4o, Aug. 6th (temp=1)	3	0	0	0	0	0.6%
Qwen 2.5 72B	3	0	0	0	0	0.6%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	85	82	80	75	67	77.8%
Minimax M2.5	91	80	80	73	56	76.2%
GPT-5 Mini	84	75	74	72	72	75.5%
Claude Sonnet 4.6	91	77	69	67	66	74.1%
GPT-5.1	88	82	72	65	62	74.0%
ByteDance Seed 1.6 Flash	79	73	72	71	69	72.7%
Z.AI GLM 5	89	83	82	59	43	71.3%
GPT-5.2	75	71	70	70	69	71.0%
Claude Opus 4	88	74	67	60	58	69.4%
Claude Opus 4.5	79	79	73	63	50	68.9%
Claude Sonnet 4.5	79	78	76	71	27	66.3%
Claude Opus 4.6	80	66	62	60	58	65.1%
ByteDance Seed 1.6	93	67	53	52	46	62.2%
Qwen 3.5 397B A17B	78	67	59	55	48	61.3%
Claude 3.7 Sonnet	85	68	62	52	39	61.1%
Claude Haiku 4.5	87	59	55	53	48	60.4%
Gemini 3 Pro (Preview)	72	59	52	51	44	55.5%
Z.AI GLM 4.7 Flash	71	61	54	47	40	54.6%
Z.AI GLM 4.7	62	58	57	51	45	54.4%
Writer: Palmyra X5	60	57	49	47	39	50.3%
GPT-4.1	63	61	43	43	35	49.0%
MoonshotAI: Kimi K2.5	72	59	53	38	23	48.9%
GPT-5 Nano	57	55	48	43	41	48.7%
Rocinante 12B	68	58	57	38	22	48.5%
Claude Sonnet 4	80	54	38	37	33	48.4%
DeepSeek V3 (2025-03-24)	78	51	49	37	19	46.8%
Llama 3.1 8B	58	55	52	52	15	46.5%
Mistral Medium 3.1	54	49	44	43	40	46.2%
Mistral Small Creative	73	42	41	40	34	45.8%
Z.AI GLM 4.5	61	51	48	47	22	45.8%
DeepSeek V3.2	58	46	46	42	33	44.8%
WizardLM 2 8x22b	54	47	46	46	29	44.6%
DeepSeek V3.1	54	54	47	39	25	44.0%
Z.AI GLM 4.6	75	43	38	32	31	43.9%
Qwen 3.5 Plus (2026-02-15)	56	49	46	39	28	43.5%
Gemini 3 Flash (Preview)	57	50	40	32	22	40.2%
Ministral 8B	58	44	33	31	19	36.8%
Mistral Large 2	59	38	37	30	20	36.8%
Arcee AI: Trinity Large (Preview)	65	55	29	21	14	36.7%
Mistral Large	58	47	44	27	0	35.2%
DeepSeek V3 (2024-12-26)	59	48	46	20	0	34.8%
Claude 3 Haiku	51	43	38	32	5	33.9%
Mistral Large 3	51	48	24	23	21	33.5%
Grok 4	53	43	27	27	14	32.7%
Gemini 3.1 Pro (Preview)	44	38	30	28	24	32.7%
Ministral 3 14B	44	43	37	36	2	32.4%
Hermes 3 405B	47	43	38	26	0	30.6%
Claude 3.5 Sonnet	65	48	38	0	0	30.4%
Gemini 2.5 Pro	49	47	29	22	0	29.7%
DeepSeek-V2 Chat	60	34	31	19	1	28.9%
Gemma 3 27B	43	38	29	29	0	27.9%
o4 Mini High	40	33	32	18	12	26.9%
Llama 3.1 Nemotron 70B	52	50	16	0	0	23.4%
Ministral 3 3B	44	27	23	17	0	22.3%
Claude 3.5 Haiku	55	45	0	0	0	20.0%
GPT-4o, May 13th (temp=0)	55	25	14	0	0	18.9%
Cohere Command R+ (Aug. 2024)	42	39	5	3	2	18.1%
Ministral 3 8B	61	26	1	0	0	17.6%
Gemma 3 12B	42	21	14	6	0	16.6%
Arcee AI: Trinity Mini	37	35	9	1	0	16.3%
Ministral 3B	47	21	6	1	0	15.1%
Hermes 3 70B	47	17	8	0	0	14.5%
Grok 4 Fast	24	21	7	6	5	12.8%
Qwen 2.5 72B	28	20	15	0	0	12.7%
Gemma 3 4B	34	13	11	5	0	12.7%
o4 Mini	26	16	8	0	0	10.0%
Grok 4.1 Fast	14	13	11	11	0	9.9%
Llama 3.1 70B	25	14	7	0	0	9.3%
Gemini 2.5 Flash Lite	22	11	3	0	0	7.2%
Gemini 2.5 Flash	27	2	0	0	0	5.9%
Mistral Small 3.2 24B	22	5	0	0	0	5.5%
Mistral NeMO	13	9	0	0	0	4.4%
GPT-4o Mini (temp=1)	19	0	0	0	0	3.8%
GPT-4o, May 13th (temp=1)	9	4	3	0	0	3.3%
Stealth: Aurora Alpha	5	0	0	0	0	1.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	67	66	66	63	58	63.9%
Claude Sonnet 4.6	70	61	56	51	50	57.7%
ByteDance Seed 1.6 Flash	73	63	52	46	41	55.1%
GPT-5 Mini	71	67	48	45	36	53.6%
Claude Haiku 4.5	78	73	50	48	18	53.5%
GPT-5.2	58	54	52	51	51	53.2%
Minimax M2.5	59	56	47	44	43	49.8%
Z.AI GLM 4.5	58	56	47	47	40	49.5%
GPT-5.1	60	51	47	39	36	46.7%
Claude Sonnet 4.5	56	53	49	44	31	46.5%
Claude Opus 4.5	59	46	46	44	37	46.3%
Claude 3.7 Sonnet	58	55	47	44	24	45.6%
Claude Opus 4.6	61	56	43	40	27	45.4%
Z.AI GLM 4.7 Flash	50	47	44	38	26	41.1%
GPT-5 Nano	61	40	39	28	25	38.7%
Hermes 3 405B	55	53	50	26	10	38.7%
Qwen 3.5 Plus (2026-02-15)	54	38	38	37	24	38.0%
Arcee AI: Trinity Large (Preview)	49	38	37	35	31	37.7%
Claude Opus 4	49	39	39	29	25	36.2%
ByteDance Seed 1.6	54	48	28	26	22	35.7%
Z.AI GLM 5	53	45	39	27	10	34.8%
MoonshotAI: Kimi K2.5	50	47	30	24	23	34.7%
Rocinante 12B	51	41	40	38	0	34.0%
Z.AI GLM 4.7	42	32	30	29	29	32.3%
Mistral Medium 3.1	45	39	30	21	19	31.0%
Mistral Small Creative	41	35	30	30	20	30.9%
Gemini 3.1 Pro (Preview)	41	36	34	32	12	30.9%
Writer: Palmyra X5	38	33	33	32	13	29.6%
Gemini 3 Pro (Preview)	35	33	30	27	13	27.7%
GPT-4.1	44	35	21	18	18	26.9%
Llama 3.1 8B	55	46	30	0	0	26.4%
Mistral Large 2	32	32	25	23	21	26.4%
Claude 3.5 Sonnet	47	29	29	21	0	25.0%
Ministral 3 8B	56	37	25	1	0	23.9%
Mistral Large 3	34	29	23	15	12	22.8%
Mistral Large	33	27	21	18	11	22.0%
Claude Sonnet 4	37	23	16	16	15	21.3%
DeepSeek V3.2	28	25	22	15	0	17.9%
Gemini 3 Flash (Preview)	39	25	22	2	0	17.7%
Hermes 3 70B	52	25	0	0	0	15.4%
Ministral 3 14B	24	24	7	6	0	12.4%
Z.AI GLM 4.6	31	19	2	0	0	10.4%
Qwen 3.5 397B A17B	25	12	9	6	0	10.3%
DeepSeek V3.1	20	14	10	0	0	8.8%
Gemini 2.5 Pro	26	17	0	0	0	8.7%
DeepSeek-V2 Chat	31	10	2	0	0	8.6%
DeepSeek V3 (2024-12-26)	16	16	0	0	0	6.4%
Grok 4	15	14	1	0	0	6.0%
Cohere Command R+ (Aug. 2024)	15	8	5	0	0	5.4%
o4 Mini	24	2	0	0	0	5.1%
Qwen 2.5 72B	25	0	0	0	0	5.1%
DeepSeek V3 (2025-03-24)	21	1	0	0	0	4.6%
Llama 3.1 70B	23	0	0	0	0	4.6%
Gemma 3 12B	20	0	0	0	0	4.1%
Mistral NeMO	20	0	0	0	0	4.0%
Gemini 2.5 Flash	12	4	4	0	0	4.0%
Claude 3 Haiku	18	0	0	0	0	3.7%
Mistral Small 3.2 24B	17	0	0	0	0	3.5%
Ministral 8B	7	6	5	0	0	3.4%
Grok 4 Fast	16	0	0	0	0	3.2%
Gemini 2.5 Flash Lite	14	0	0	0	0	2.8%
GPT-4o, Aug. 6th (temp=1)	12	0	0	0	0	2.4%
Ministral 3B	5	3	2	0	0	2.0%
Ministral 3 3B	8	0	0	0	0	1.6%
Grok 4.1 Fast	7	0	0	0	0	1.4%
Gemma 3 27B	2	1	0	0	0	0.6%
Claude 3.5 Haiku	3	0	0	0	0	0.6%
o4 Mini High	3	0	0	0	0	0.6%
GPT-4o, May 13th (temp=0)	2	0	0	0	0	0.3%
Arcee AI: Trinity Mini	1	0	0	0	0	0.1%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=1)	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	75	75	74	73	69	72.9%
Claude Opus 4.6	75	67	63	53	50	61.5%
Claude Sonnet 4.6	63	54	49	49	39	50.8%
ByteDance Seed 1.6 Flash	61	56	51	48	36	50.5%
GPT-5 Mini	58	54	53	41	38	48.8%
Minimax M2.5	62	54	53	36	30	47.1%
GPT-5.2	51	47	46	44	32	43.8%
GPT-5 Nano	46	46	44	41	36	42.5%
GPT-5.1	45	41	41	40	38	41.1%
Claude Opus 4	58	57	40	36	9	40.1%
Claude Haiku 4.5	55	41	35	34	31	39.3%
Gemini 3 Pro (Preview)	56	46	39	37	14	38.4%
Z.AI GLM 5	50	44	39	34	23	37.9%
Mistral Medium 3.1	54	39	33	31	29	37.1%
Llama 3.1 8B	61	49	37	19	18	36.7%
GPT-4.1	68	36	31	25	18	35.6%
Gemini 3 Flash (Preview)	46	35	35	33	26	34.9%
Qwen 3.5 397B A17B	45	42	40	31	17	34.8%
Claude 3.7 Sonnet	44	41	38	38	11	34.3%
Claude Sonnet 4.5	73	46	27	15	2	32.6%
MoonshotAI: Kimi K2.5	46	42	27	25	16	31.2%
Z.AI GLM 4.7 Flash	43	33	32	27	18	30.7%
Z.AI GLM 4.7	33	31	29	28	21	28.3%
Gemini 3.1 Pro (Preview)	41	35	29	23	14	28.3%
Claude Opus 4.5	52	33	33	12	8	27.8%
Rocinante 12B	48	40	27	10	1	25.3%
DeepSeek V3.2	38	33	25	19	10	24.9%
DeepSeek V3 (2025-03-24)	34	34	32	19	0	23.9%
Z.AI GLM 4.5	53	37	10	9	0	22.0%
Ministral 3 14B	42	26	24	9	2	20.5%
Qwen 2.5 72B	100	0	0	0	0	20.0%
Qwen 3.5 Plus (2026-02-15)	36	30	15	12	5	19.8%
ByteDance Seed 1.6	39	33	9	8	8	19.6%
Writer: Palmyra X5	36	30	25	4	0	18.9%
Z.AI GLM 4.6	47	21	17	8	1	18.6%
Claude 3.5 Sonnet	35	32	12	8	0	17.5%
Mistral Small Creative	31	24	17	10	4	17.2%
Mistral Large 2	39	15	12	9	0	15.0%
DeepSeek V3.1	29	20	17	1	0	13.5%
Arcee AI: Trinity Large (Preview)	45	11	9	2	0	13.5%
Hermes 3 70B	29	14	7	0	0	10.1%
Gemma 3 27B	33	8	0	0	0	8.1%
Mistral Large 3	29	6	4	0	0	7.8%
Claude Sonnet 4	18	14	2	0	0	6.8%
Gemini 2.5 Pro	13	9	8	0	0	6.3%
Grok 4 Fast	29	2	0	0	0	6.3%
Hermes 3 405B	27	3	0	0	0	6.0%
Claude 3.5 Haiku	26	0	0	0	0	5.3%
Ministral 3B	18	6	0	0	0	5.0%
Mistral Large	19	5	0	0	0	4.8%
o4 Mini	13	7	1	0	0	4.0%
Grok 4.1 Fast	12	5	0	0	0	3.3%
Ministral 3 8B	12	3	0	0	0	3.0%
Ministral 8B	6	5	4	0	0	3.0%
Grok 4	12	0	0	0	0	2.4%
Gemma 3 12B	9	0	0	0	0	1.8%
Cohere Command R+ (Aug. 2024)	9	0	0	0	0	1.8%
DeepSeek V3 (2024-12-26)	9	0	0	0	0	1.7%
Ministral 3 3B	7	0	0	0	0	1.4%
Mistral Small 3.2 24B	7	0	0	0	0	1.4%
o4 Mini High	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
DeepSeek-V2 Chat	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=1)	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
ByteDance Seed 1.6 Flash	87	87	85	78	68	81.1%
GPT-5	82	81	81	80	78	80.4%
Claude Sonnet 4.6	85	81	76	76	75	78.6%
GPT-5.1	81	79	75	74	74	76.5%
GPT-5 Mini	80	79	75	75	59	73.6%
Claude Opus 4.6	76	69	66	65	62	67.5%
Minimax M2.5	73	72	70	68	49	66.6%
Claude Haiku 4.5	89	65	63	59	56	66.3%
GPT-5.2	76	70	68	66	48	65.5%
ByteDance Seed 1.6	70	69	64	63	55	64.1%
MoonshotAI: Kimi K2.5	72	71	62	54	54	62.5%
Claude Opus 4.5	69	68	64	59	52	62.1%
Claude Sonnet 4.5	71	64	64	58	51	61.6%
Writer: Palmyra X5	68	63	61	57	55	61.0%
Ministral 3 8B	82	71	59	44	44	60.1%
Claude Opus 4	64	61	58	58	55	59.1%
GPT-4.1	69	64	61	61	39	58.9%
Z.AI GLM 5	65	63	59	54	46	57.3%
Claude 3.5 Sonnet	65	65	54	52	48	56.8%
Z.AI GLM 4.7	71	56	56	54	43	56.0%
Ministral 8B	72	64	57	47	37	55.5%
Claude 3.7 Sonnet	69	68	55	47	38	55.4%
GPT-5 Nano	70	63	49	46	42	54.1%
Gemini 3 Flash (Preview)	60	57	56	49	48	54.0%
Mistral Large	68	61	54	48	30	52.4%
DeepSeek V3 (2025-03-24)	78	59	56	48	19	52.0%
Ministral 3 14B	58	53	51	47	47	51.4%
Gemini 3 Pro (Preview)	60	60	46	42	41	49.7%
Gemini 2.5 Pro	57	55	47	45	42	49.3%
Arcee AI: Trinity Large (Preview)	62	60	46	44	32	48.8%
DeepSeek V3.2	56	56	53	41	35	48.3%
Mistral Medium 3.1	68	61	49	30	30	47.5%
Z.AI GLM 4.5	76	56	38	35	24	46.0%
Claude Sonnet 4	57	48	44	42	35	45.3%
Mistral Large 2	69	57	45	32	20	44.5%
Z.AI GLM 4.7 Flash	54	52	46	39	32	44.5%
Z.AI GLM 4.6	52	48	44	38	37	43.8%
o4 Mini	65	51	48	32	15	42.0%
Gemini 3.1 Pro (Preview)	50	50	50	34	26	41.8%
Mistral Small Creative	64	53	49	39	4	41.8%
DeepSeek V3.1	48	46	41	40	33	41.6%
Mistral Large 3	49	46	45	44	22	41.2%
Rocinante 12B	68	59	57	21	0	41.0%
Qwen 3.5 397B A17B	62	51	36	25	24	39.7%
Qwen 3.5 Plus (2026-02-15)	52	45	36	21	19	34.8%
o4 Mini High	48	43	35	25	22	34.5%
Llama 3.1 8B	59	55	54	0	0	33.5%
DeepSeek V3 (2024-12-26)	57	50	25	10	0	28.3%
Hermes 3 405B	58	38	24	11	10	28.2%
Gemma 3 27B	59	29	19	18	0	25.1%
Grok 4 Fast	35	24	21	20	19	23.7%
Grok 4	33	29	24	15	11	22.3%
Llama 3.1 70B	60	36	13	0	0	21.7%
Claude 3.5 Haiku	31	31	22	22	0	21.2%
Ministral 3 3B	53	33	15	3	0	20.8%
Ministral 3B	31	28	27	18	0	20.7%
Gemma 3 12B	25	24	23	20	12	20.7%
Hermes 3 70B	36	35	26	5	0	20.5%
Grok 4.1 Fast	36	26	21	12	7	20.5%
GPT-4.1 Mini	37	29	14	9	0	17.7%
Mistral Small 3.2 24B	43	41	0	0	0	16.9%
DeepSeek-V2 Chat	60	11	9	3	0	16.6%
Gemini 2.5 Flash Lite	38	23	9	6	1	15.4%
Gemini 2.5 Flash	27	23	21	3	0	14.9%
Mistral NeMO	31	25	17	1	0	14.9%
WizardLM 2 8x22b	38	29	0	0	0	13.5%
GPT-4o, May 13th (temp=1)	43	17	0	0	0	12.1%
Arcee AI: Trinity Mini	25	25	0	0	0	9.9%
Stealth: Aurora Alpha	21	15	8	4	0	9.8%
Cohere Command R+ (Aug. 2024)	17	13	7	4	0	8.1%
Gemma 3 4B	17	9	8	0	0	6.9%
GPT-4.1 Nano	33	0	0	0	0	6.6%
Claude 3 Haiku	23	3	0	0	0	5.2%
Qwen 2.5 72B	9	1	0	0	0	2.0%
GPT-4o Mini (temp=1)	3	0	0	0	0	0.7%
Llama 3.1 Nemotron 70B	2	0	0	0	0	0.3%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	91	79	77	76	74	79.4%
GPT-5 Mini	85	80	79	76	72	78.4%
GPT-5	85	76	75	72	68	75.3%
Claude Opus 4.5	89	81	70	69	67	75.1%
Claude Opus 4.6	83	77	76	72	66	74.8%
GPT-5.2	77	73	73	72	64	71.7%
GPT-5.1	72	72	70	64	58	67.0%
ByteDance Seed 1.6	79	72	67	60	56	66.8%
ByteDance Seed 1.6 Flash	80	68	64	61	60	66.7%
Z.AI GLM 4.5	78	75	66	58	56	66.3%
Z.AI GLM 5	76	68	63	61	54	64.3%
MoonshotAI: Kimi K2.5	73	66	62	57	53	62.2%
Claude Haiku 4.5	79	74	66	48	43	61.9%
GPT-5 Nano	70	61	61	59	56	61.4%
Claude Opus 4	75	64	62	56	46	60.5%
Minimax M2.5	73	69	62	48	47	60.0%
Gemini 3 Pro (Preview)	70	66	58	55	50	59.8%
Claude 3.7 Sonnet	66	64	64	62	42	59.6%
Claude Sonnet 4.5	82	64	48	47	46	57.4%
Claude 3.5 Sonnet	87	55	53	53	37	56.8%
Qwen 3.5 397B A17B	63	61	56	51	51	56.4%
Gemini 3.1 Pro (Preview)	66	65	63	44	40	55.4%
Rocinante 12B	72	65	65	51	24	55.3%
DeepSeek-V2 Chat	76	56	55	52	33	54.3%
Claude Sonnet 4	63	61	56	50	35	53.0%
Hermes 3 405B	79	52	50	40	36	51.3%
Ministral 3 8B	70	70	62	34	21	51.3%
Hermes 3 70B	75	67	39	37	36	50.9%
Gemini 3 Flash (Preview)	54	52	51	45	44	49.1%
Ministral 3 14B	65	65	42	37	30	47.9%
Z.AI GLM 4.7 Flash	54	51	50	41	39	47.3%
Mistral Large 2	65	60	47	39	25	47.1%
Qwen 3.5 Plus (2026-02-15)	60	59	53	34	29	47.1%
Gemini 2.5 Pro	54	52	48	47	34	47.0%
Arcee AI: Trinity Large (Preview)	65	60	46	33	29	46.6%
Mistral Large 3	57	54	54	41	27	46.5%
Mistral Medium 3.1	60	53	46	40	34	46.4%
Writer: Palmyra X5	60	53	48	40	32	46.3%
Mistral Large	53	49	48	47	33	46.1%
Z.AI GLM 4.7	56	50	47	40	31	45.0%
Mistral Small Creative	57	45	40	36	36	42.8%
DeepSeek V3.2	58	50	41	34	30	42.6%
Z.AI GLM 4.6	59	51	43	36	18	41.5%
Gemini 2.5 Flash	56	51	43	36	14	40.2%
DeepSeek V3.1	54	49	41	41	13	39.8%
GPT-4.1	59	47	38	29	23	39.1%
Llama 3.1 8B	64	63	33	20	15	38.8%
Ministral 8B	54	48	27	27	27	36.8%
Ministral 3B	52	45	35	30	20	36.4%
Grok 4	47	43	33	26	23	34.3%
DeepSeek V3 (2025-03-24)	39	37	33	32	16	31.4%
Grok 4.1 Fast	48	35	32	24	14	30.6%
DeepSeek V3 (2024-12-26)	54	49	27	12	7	29.9%
Gemma 3 4B	54	38	32	14	13	29.9%
Llama 3.1 70B	66	65	16	0	0	29.2%
GPT-4.1 Mini	63	33	32	15	0	28.6%
Grok 4 Fast	42	33	33	17	12	27.4%
Cohere Command R+ (Aug. 2024)	65	61	10	0	0	27.0%
o4 Mini	36	32	30	30	2	26.1%
Gemma 3 27B	42	40	28	19	2	26.0%
GPT-4o, May 13th (temp=0)	48	34	20	16	4	24.5%
GPT-4o, May 13th (temp=1)	37	34	18	17	7	22.7%
Ministral 3 3B	39	31	25	16	0	22.1%
Gemma 3 12B	57	20	17	14	1	21.8%
Claude 3 Haiku	39	35	21	10	0	21.0%
Arcee AI: Trinity Mini	39	26	25	13	0	20.6%
o4 Mini High	39	30	17	9	0	19.0%
Mistral NeMO	29	23	22	15	7	19.0%
Stealth: Aurora Alpha	26	20	18	14	12	18.0%
GPT-4o Mini (temp=1)	36	32	16	0	0	16.7%
Gemini 2.5 Flash Lite	39	22	14	5	1	16.1%
Mistral Small 3.2 24B	29	21	21	4	0	14.8%
Llama 3.1 Nemotron 70B	30	15	14	13	0	14.5%
GPT-4.1 Nano	30	14	14	11	0	13.7%
Qwen 2.5 72B	27	22	11	9	0	13.7%
WizardLM 2 8x22b	29	21	17	0	0	13.4%
GPT-4o, Aug. 6th (temp=0)	25	19	9	3	0	11.1%
GPT-4o, Aug. 6th (temp=1)	21	13	6	1	0	8.2%
Claude 3.5 Haiku	18	0	0	0	0	3.6%
GPT-4o Mini (temp=0)	13	1	0	0	0	3.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	91	91	90	88	64	84.9%
ByteDance Seed 1.6 Flash	91	87	85	83	76	84.4%
GPT-5	81	78	77	76	71	76.7%
GPT-5 Mini	81	78	73	73	69	75.0%
Claude Opus 4.6	84	76	70	70	64	72.8%
ByteDance Seed 1.6	91	75	69	65	53	70.5%
GPT-5.1	88	71	67	60	59	69.0%
Claude Haiku 4.5	71	70	68	62	51	64.5%
Minimax M2.5	74	71	69	57	49	64.1%
Arcee AI: Trinity Large (Preview)	87	69	60	53	47	63.1%
Claude Opus 4.5	71	70	64	62	47	62.8%
Claude Sonnet 4.5	77	68	65	64	39	62.6%
Writer: Palmyra X5	73	71	65	54	44	61.3%
Claude Opus 4	75	60	60	60	48	60.5%
Z.AI GLM 5	73	64	56	55	53	60.1%
GPT-5.2	66	64	62	59	49	60.1%
DeepSeek V3 (2025-03-24)	100	59	47	41	40	57.4%
Claude Sonnet 4	82	58	57	54	35	57.2%
Mistral Large	71	70	57	49	33	55.9%
GPT-5 Nano	61	59	59	48	48	55.1%
Qwen 3.5 397B A17B	66	64	58	48	35	54.2%
Mistral Large 3	69	69	52	42	40	54.1%
Rocinante 12B	91	60	49	43	23	53.3%
Llama 3.1 8B	63	54	54	53	41	53.1%
Claude 3.5 Sonnet	63	57	55	47	38	51.9%
GPT-4.1	63	61	50	40	39	50.4%
Claude 3.7 Sonnet	60	54	52	48	39	50.4%
Ministral 3 14B	65	64	56	33	32	50.0%
Gemini 3 Pro (Preview)	63	53	48	45	41	49.9%
Z.AI GLM 4.7	57	56	52	46	38	49.8%
Z.AI GLM 4.5	59	55	53	41	33	48.3%
Gemini 3 Flash (Preview)	67	49	47	45	32	48.0%
Mistral Large 2	60	51	51	46	32	48.0%
MoonshotAI: Kimi K2.5	57	56	48	45	33	47.9%
Claude 3.5 Haiku	78	63	53	26	18	47.8%
Ministral 3 8B	71	65	40	34	26	47.3%
Qwen 3.5 Plus (2026-02-15)	59	51	45	38	34	45.3%
Hermes 3 70B	68	49	43	38	24	44.4%
Z.AI GLM 4.7 Flash	57	46	41	39	37	44.1%
Mistral Medium 3.1	49	48	48	45	30	43.9%
DeepSeek V3.2	53	49	45	38	26	42.4%
Gemini 3.1 Pro (Preview)	48	42	42	36	24	38.4%
Mistral Small Creative	53	48	40	28	22	38.3%
Z.AI GLM 4.6	52	41	29	27	19	33.7%
DeepSeek V3 (2024-12-26)	54	45	39	15	4	31.5%
DeepSeek V3.1	49	32	31	25	17	30.6%
Ministral 3 3B	47	40	30	18	14	30.0%
DeepSeek-V2 Chat	47	38	30	24	12	30.0%
Grok 4	43	35	32	22	18	29.8%
Gemini 2.5 Pro	47	34	25	22	18	29.1%
Hermes 3 405B	54	48	18	10	9	27.7%
o4 Mini High	36	29	26	19	16	25.3%
Grok 4 Fast	38	27	27	24	9	25.1%
Ministral 3B	48	29	18	17	12	25.1%
Mistral NeMO	55	28	17	13	12	24.9%
Ministral 8B	44	30	28	22	0	24.9%
Cohere Command R+ (Aug. 2024)	69	37	19	0	0	24.9%
o4 Mini	35	27	21	21	16	23.8%
GPT-4o, May 13th (temp=0)	34	31	29	9	8	22.2%
GPT-4.1 Mini	61	42	2	0	0	20.9%
Gemma 3 27B	50	26	15	9	2	20.2%
Grok 4.1 Fast	35	23	17	17	5	19.4%
Mistral Small 3.2 24B	45	24	21	1	0	18.3%
Arcee AI: Trinity Mini	54	24	11	0	0	17.8%
Llama 3.1 70B	63	25	0	0	0	17.6%
Gemini 2.5 Flash	32	23	14	11	5	17.0%
Gemini 2.5 Flash Lite	30	18	11	0	0	11.9%
Qwen 2.5 72B	48	10	0	0	0	11.6%
GPT-4.1 Nano	18	17	10	0	0	9.0%
Stealth: Aurora Alpha	19	17	5	0	0	8.2%
GPT-4o, Aug. 6th (temp=1)	23	12	4	1	0	8.1%
Gemma 3 4B	20	5	0	0	0	4.9%
Gemma 3 12B	16	7	0	0	0	4.6%
GPT-4o, Aug. 6th (temp=0)	16	0	0	0	0	3.1%
GPT-4o, May 13th (temp=1)	15	0	0	0	0	3.0%
Claude 3 Haiku	3	0	0	0	0	0.5%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
WizardLM 2 8x22b	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	100	85	77	77	76	83.1%
GPT-5	81	80	79	74	73	77.1%
Minimax M2.5	81	76	74	70	66	73.7%
GPT-5 Mini	83	75	69	66	66	72.1%
Claude Opus 4.5	77	73	69	67	67	70.5%
Claude Haiku 4.5	77	74	70	67	55	68.6%
Claude Opus 4	88	72	62	61	55	67.7%
ByteDance Seed 1.6 Flash	88	81	71	50	48	67.6%
Claude Sonnet 4.5	78	71	64	63	60	67.4%
ByteDance Seed 1.6	85	70	69	61	47	66.2%
GPT-5.1	74	68	68	57	55	64.5%
Z.AI GLM 5	81	68	67	56	45	63.3%
Z.AI GLM 4.7	77	74	65	54	39	61.6%
Claude 3.7 Sonnet	69	64	62	59	51	61.0%
WizardLM 2 8x22b	100	100	66	28	10	60.9%
GPT-5.2	79	66	61	57	36	59.9%
Claude Sonnet 4	75	71	59	48	45	59.6%
Claude Opus 4.6	68	62	57	56	48	58.0%
DeepSeek V3 (2025-03-24)	83	57	55	55	39	57.7%
MoonshotAI: Kimi K2.5	78	61	60	46	42	57.4%
Claude 3.5 Sonnet	72	67	58	55	29	56.4%
Rocinante 12B	92	61	50	44	31	55.6%
Llama 3.1 8B	70	61	55	49	41	55.4%
Gemini 3 Flash (Preview)	67	57	52	44	43	52.6%
Gemini 3 Pro (Preview)	67	64	53	45	32	52.1%
Z.AI GLM 4.5	73	60	51	41	35	51.9%
Z.AI GLM 4.7 Flash	69	57	48	46	33	50.3%
Qwen 3.5 397B A17B	59	57	46	41	41	48.9%
DeepSeek-V2 Chat	74	50	47	39	33	48.7%
Ministral 3 14B	61	56	48	44	32	48.2%
Hermes 3 70B	67	61	47	37	29	48.2%
Mistral Large 3	70	58	52	28	23	46.3%
GPT-5 Nano	55	47	45	43	39	45.8%
Arcee AI: Trinity Large (Preview)	61	59	55	31	22	45.4%
DeepSeek V3.1	61	55	37	36	29	43.6%
DeepSeek V3.2	50	49	48	40	25	42.5%
Gemini 3.1 Pro (Preview)	52	52	43	37	27	42.1%
Writer: Palmyra X5	61	53	52	27	10	40.6%
Qwen 3.5 Plus (2026-02-15)	51	48	37	36	31	40.6%
Mistral Medium 3.1	50	47	37	34	31	39.7%
Mistral Large 2	57	46	43	32	17	39.0%
Ministral 3 8B	50	46	37	33	28	38.8%
Mistral Small Creative	51	49	34	34	26	38.6%
Gemini 2.5 Pro	62	49	31	27	18	37.2%
DeepSeek V3 (2024-12-26)	62	38	37	26	6	33.8%
Ministral 8B	51	46	32	22	11	32.4%
Z.AI GLM 4.6	50	40	31	21	17	31.9%
Gemma 3 27B	38	37	33	29	19	31.1%
Mistral Large	54	28	26	20	20	29.7%
Gemma 3 4B	51	48	30	11	8	29.4%
o4 Mini	48	43	27	21	1	28.1%
GPT-4.1	43	39	24	23	11	27.8%
o4 Mini High	41	30	26	21	18	27.2%
Hermes 3 405B	61	44	27	0	0	26.5%
Gemini 2.5 Flash	46	41	29	14	0	26.0%
Grok 4.1 Fast	49	22	21	21	17	25.9%
Ministral 3B	60	32	28	4	0	24.9%
Grok 4	47	34	29	7	0	23.4%
Gemini 2.5 Flash Lite	60	33	13	11	0	23.3%
Llama 3.1 70B	37	33	22	18	0	21.9%
Claude 3.5 Haiku	43	40	16	9	0	21.6%
Ministral 3 3B	58	23	11	8	0	20.1%
Cohere Command R+ (Aug. 2024)	52	12	7	0	0	14.2%
Grok 4 Fast	20	19	15	11	0	13.1%
GPT-4o, May 13th (temp=0)	48	17	0	0	0	12.9%
Arcee AI: Trinity Mini	20	18	13	0	0	10.1%
Llama 3.1 Nemotron 70B	24	20	5	0	0	9.7%
Gemma 3 12B	24	13	7	2	0	9.2%
GPT-4o, May 13th (temp=1)	30	13	0	0	0	8.6%
Claude 3 Haiku	32	7	3	0	0	8.5%
GPT-4.1 Nano	25	14	2	0	0	8.2%
Mistral NeMO	29	3	2	2	0	7.4%
Qwen 2.5 72B	28	0	0	0	0	5.5%
GPT-4o, Aug. 6th (temp=0)	15	0	0	0	0	3.0%
GPT-4.1 Mini	8	6	0	0	0	2.8%
GPT-4o, Aug. 6th (temp=1)	12	0	0	0	0	2.3%
GPT-4o Mini (temp=1)	5	0	0	0	0	1.1%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
ByteDance Seed 1.6 Flash	65	64	62	58	51	60.2%
GPT-5	73	68	63	54	39	59.5%
GPT-5.1	59	57	53	51	39	52.0%
Claude Sonnet 4.6	59	55	50	42	39	49.0%
GPT-5 Mini	55	54	48	46	32	47.1%
GPT-5.2	59	52	45	42	38	47.0%
Claude Opus 4.6	56	51	48	38	33	45.3%
Z.AI GLM 5	63	56	47	34	25	45.1%
Claude Haiku 4.5	62	60	50	28	20	44.2%
Claude 3.7 Sonnet	60	44	41	39	27	42.0%
Writer: Palmyra X5	54	43	42	42	20	40.6%
Llama 3.1 8B	64	47	42	42	0	39.1%
DeepSeek V3.2	52	48	41	31	17	37.8%
Rocinante 12B	100	43	23	15	0	36.4%
Z.AI GLM 4.7 Flash	54	50	41	22	12	35.8%
Minimax M2.5	58	51	29	22	17	35.6%
Ministral 3 14B	50	43	36	26	21	35.0%
Claude Opus 4.5	51	44	43	24	13	34.9%
Mistral Medium 3.1	43	42	37	27	23	34.3%
Claude Sonnet 4.5	40	37	36	35	21	33.9%
Z.AI GLM 4.5	60	45	32	17	8	32.3%
GPT-5 Nano	40	36	32	29	24	32.3%
ByteDance Seed 1.6	59	42	27	17	17	32.3%
Qwen 3.5 Plus (2026-02-15)	49	48	34	28	0	32.1%
Mistral Large	52	32	28	23	9	28.8%
Mistral Small Creative	37	32	31	19	9	25.6%
Gemini 3 Pro (Preview)	40	35	34	18	0	25.5%
MoonshotAI: Kimi K2.5	43	35	31	7	3	23.9%
Claude Opus 4	46	25	25	20	0	23.2%
Z.AI GLM 4.7	49	42	14	11	0	23.1%
Claude 3.5 Sonnet	50	32	31	0	0	22.5%
Claude Sonnet 4	42	29	16	14	0	20.2%
Qwen 3.5 397B A17B	31	22	16	15	13	19.3%
Ministral 8B	40	27	15	13	0	19.2%
Cohere Command R+ (Aug. 2024)	41	32	17	3	0	18.6%
Arcee AI: Trinity Large (Preview)	37	35	11	9	0	18.5%
Hermes 3 70B	47	22	18	0	0	17.3%
DeepSeek V3 (2025-03-24)	31	30	16	6	0	16.6%
Z.AI GLM 4.6	25	19	18	0	0	12.3%
Mistral Large 3	24	24	9	0	0	11.5%
WizardLM 2 8x22b	22	15	7	6	6	11.1%
GPT-4.1	31	15	9	0	0	11.1%
Gemini 3.1 Pro (Preview)	19	12	11	9	0	10.0%
Mistral Large 2	26	12	9	0	0	9.4%
Ministral 3 8B	33	13	1	0	0	9.4%
Gemini 2.5 Pro	17	15	14	0	0	9.2%
Hermes 3 405B	24	19	3	0	0	9.0%
Gemini 3 Flash (Preview)	23	16	6	0	0	8.8%
DeepSeek-V2 Chat	21	12	5	0	0	7.6%
o4 Mini	17	8	3	1	0	5.9%
Ministral 3B	19	10	0	0	0	5.9%
Ministral 3 3B	15	9	0	0	0	4.9%
DeepSeek V3.1	18	6	0	0	0	4.8%
Gemma 3 27B	14	6	0	0	0	4.0%
Mistral NeMO	12	0	0	0	0	2.5%
Grok 4 Fast	11	1	1	0	0	2.5%
Qwen 2.5 72B	7	3	0	0	0	2.1%
DeepSeek V3 (2024-12-26)	6	3	0	0	0	1.7%
GPT-4.1 Mini	8	0	0	0	0	1.5%
o4 Mini High	6	0	0	0	0	1.1%
Gemini 2.5 Flash Lite	5	0	0	0	0	1.1%
Gemini 2.5 Flash	4	0	0	0	0	0.8%
Grok 4	4	0	0	0	0	0.8%
Claude 3.5 Haiku	2	0	0	0	0	0.4%
Grok 4.1 Fast	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Gemma 3 12B	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%
Gemma 3 4B	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	77	67	66	63	59	66.4%
ByteDance Seed 1.6 Flash	60	58	51	46	38	50.6%
Llama 3.1 8B	71	54	54	49	24	50.5%
Minimax M2.5	64	55	46	44	37	49.1%
GPT-5.1	59	56	52	46	32	49.0%
Claude Sonnet 4.6	74	53	48	45	14	46.7%
GPT-5.2	53	49	45	40	39	45.1%
Claude Opus 4.6	59	55	40	30	29	42.6%
GPT-5 Mini	58	49	48	34	24	42.5%
Z.AI GLM 5	50	43	43	40	30	41.3%
Z.AI GLM 4.7 Flash	53	40	39	35	24	38.2%
MoonshotAI: Kimi K2.5	56	54	52	25	0	37.4%
ByteDance Seed 1.6	56	43	42	31	13	37.1%
Claude Opus 4.5	55	40	38	32	19	36.7%
Z.AI GLM 4.7	56	40	38	21	15	33.9%
GPT-4.1	48	38	29	28	24	33.5%
Claude 3.7 Sonnet	57	48	27	26	9	33.4%
GPT-5 Nano	47	42	30	25	22	33.3%
Gemini 3 Flash (Preview)	39	37	34	23	19	30.4%
Gemini 3 Pro (Preview)	42	41	25	17	17	28.4%
Claude Haiku 4.5	43	40	26	19	6	26.5%
Claude Opus 4	62	23	18	17	9	25.7%
Qwen 3.5 397B A17B	52	32	29	13	0	25.0%
Claude Sonnet 4.5	49	27	22	21	0	23.8%
Mistral Large 3	43	40	24	13	0	23.8%
Writer: Palmyra X5	40	34	14	11	0	20.0%
Mistral Small 3.2 24B	97	0	0	0	0	19.5%
Mistral Medium 3.1	28	21	20	19	0	17.8%
Mistral Large	51	14	10	7	0	16.5%
Qwen 3.5 Plus (2026-02-15)	33	19	19	9	0	16.2%
Rocinante 12B	26	20	18	10	0	14.8%
Gemini 3.1 Pro (Preview)	42	26	0	0	0	13.7%
WizardLM 2 8x22b	38	15	8	7	0	13.7%
Z.AI GLM 4.5	27	25	0	0	0	10.4%
DeepSeek V3.2	38	10	2	0	0	10.0%
Claude Sonnet 4	26	17	4	0	0	9.4%
Gemma 3 27B	25	15	7	0	0	9.4%
DeepSeek V3.1	42	2	0	0	0	8.7%
DeepSeek V3 (2025-03-24)	16	11	9	5	0	8.1%
Grok 4 Fast	27	10	0	0	0	7.5%
Z.AI GLM 4.6	24	6	5	0	0	7.1%
Ministral 3 8B	34	2	0	0	0	7.1%
Hermes 3 70B	18	13	3	0	0	6.8%
Grok 4.1 Fast	16	13	4	0	0	6.5%
Claude 3.5 Sonnet	28	2	0	0	0	6.0%
Arcee AI: Trinity Mini	29	0	0	0	0	5.8%
DeepSeek-V2 Chat	24	0	0	0	0	4.8%
Mistral Small Creative	12	10	1	0	0	4.7%
Arcee AI: Trinity Large (Preview)	9	8	5	1	0	4.5%
Mistral Large 2	18	4	0	0	0	4.5%
Ministral 8B	8	6	5	0	0	3.8%
Hermes 3 405B	17	0	0	0	0	3.3%
Ministral 3B	16	0	0	0	0	3.1%
DeepSeek V3 (2024-12-26)	9	3	0	0	0	2.5%
Ministral 3 14B	7	0	0	0	0	1.5%
Claude 3 Haiku	7	0	0	0	0	1.3%
Gemini 2.5 Pro	3	2	0	0	0	1.1%
Gemma 3 12B	5	0	0	0	0	0.9%
o4 Mini High	5	0	0	0	0	0.9%
o4 Mini	4	0	0	0	0	0.9%
Gemma 3 4B	4	0	0	0	0	0.7%
Qwen 2.5 72B	3	0	0	0	0	0.6%
Cohere Command R+ (Aug. 2024)	3	0	0	0	0	0.5%
Grok 4	1	0	0	0	0	0.3%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o, May 13th (temp=1)	0	0	0	0	0	0.0%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
GPT-4.1 Mini	0	0	0	0	0	0.0%
Gemini 2.5 Flash	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 70B	0	0	0	0	0	0.0%
Gemini 2.5 Flash Lite	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%
Ministral 3 3B	0	0	0	0	0	0.0%
Mistral NeMO	0	0	0	0	0	0.0%
GPT-4.1 Nano	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5	86	86	84	83	81	83.9%
Claude Sonnet 4.6	91	86	81	77	76	82.2%
GPT-5.1	86	82	80	79	74	80.2%
Claude Opus 4.6	89	84	82	73	66	78.5%
ByteDance Seed 1.6 Flash	93	82	74	70	70	78.1%
GPT-5 Mini	83	81	80	74	63	76.2%
GPT-5.2	79	77	73	71	70	73.8%
Claude Haiku 4.5	79	73	72	72	72	73.4%
Z.AI GLM 5	79	77	73	66	61	71.3%
Claude Opus 4.5	75	72	71	70	65	70.5%
Qwen 3.5 397B A17B	73	72	71	71	61	69.6%
Minimax M2.5	81	74	68	65	55	68.8%
Claude Sonnet 4.5	78	69	69	64	58	67.8%
Writer: Palmyra X5	79	69	63	60	53	64.7%
Claude Opus 4	69	65	63	60	55	62.3%
GPT-5 Nano	61	60	58	58	56	58.5%
Mistral Large	70	58	57	50	50	57.3%
ByteDance Seed 1.6	71	65	64	54	30	57.0%
Gemini 3 Pro (Preview)	61	58	57	56	48	56.0%
Claude 3.7 Sonnet	71	56	50	47	47	54.2%
DeepSeek V3.2	72	61	59	47	32	54.1%
Qwen 3.5 Plus (2026-02-15)	67	61	58	44	36	53.1%
WizardLM 2 8x22b	63	62	49	43	43	52.2%
GPT-4.1	64	60	55	43	33	50.9%
MoonshotAI: Kimi K2.5	68	51	51	47	35	50.3%
Z.AI GLM 4.6	64	59	53	45	29	50.2%
Gemini 3.1 Pro (Preview)	72	61	58	33	27	50.2%
Gemini 3 Flash (Preview)	61	54	54	39	38	49.0%
Mistral Large 3	74	66	57	32	14	48.6%
Hermes 3 70B	67	49	47	43	33	47.8%
Claude 3.5 Sonnet	59	55	48	39	36	47.4%
DeepSeek V3 (2025-03-24)	64	52	45	37	32	46.1%
Z.AI GLM 4.7 Flash	55	50	48	45	31	45.9%
Ministral 3 14B	72	59	46	42	10	45.6%
Z.AI GLM 4.7	69	63	38	31	26	45.4%
Mistral Small Creative	62	55	45	36	22	44.0%
Ministral 8B	66	61	61	20	11	43.7%
Mistral Large 2	69	44	39	37	25	42.8%
Z.AI GLM 4.5	54	48	38	37	35	42.3%
Mistral Medium 3.1	55	52	43	36	23	41.8%
Arcee AI: Trinity Large (Preview)	67	48	43	43	4	41.3%
DeepSeek V3.1	66	45	44	25	23	40.6%
Claude Sonnet 4	51	46	43	36	27	40.6%
o4 Mini High	58	48	42	41	0	37.7%
Grok 4.1 Fast	45	44	34	32	23	35.8%
Cohere Command R+ (Aug. 2024)	71	39	34	30	0	34.7%
Gemma 3 27B	43	35	31	23	18	30.1%
Hermes 3 405B	54	43	24	21	9	30.1%
Grok 4 Fast	41	37	35	20	17	30.0%
Grok 4	40	35	27	24	22	29.7%
Gemini 2.5 Pro	41	39	37	16	14	29.4%
GPT-4.1 Mini	49	42	28	27	0	29.4%
Mistral NeMO	58	43	43	0	0	28.8%
Llama 3.1 8B	52	39	37	16	0	28.6%
DeepSeek V3 (2024-12-26)	64	32	32	8	0	27.1%
o4 Mini	34	28	28	24	12	25.2%
Ministral 3 8B	42	29	21	14	10	23.4%
Ministral 3B	48	35	33	0	0	23.3%
GPT-4.1 Nano	35	26	23	20	9	22.5%
DeepSeek-V2 Chat	41	26	22	21	0	22.1%
Rocinante 12B	50	32	24	0	0	21.1%
Qwen 2.5 72B	46	37	18	0	0	20.2%
Gemma 3 12B	30	24	16	5	0	15.1%
GPT-4o, Aug. 6th (temp=1)	31	16	13	6	0	13.2%
Gemini 2.5 Flash Lite	22	22	0	0	0	8.7%
Llama 3.1 70B	38	0	0	0	0	7.5%
Stealth: Aurora Alpha	22	13	1	0	0	7.1%
Gemma 3 4B	17	16	2	0	0	7.0%
Ministral 3 3B	28	7	0	0	0	6.9%
Gemini 2.5 Flash	17	17	0	0	0	6.8%
Mistral Small 3.2 24B	29	2	0	0	0	6.2%
GPT-4o, May 13th (temp=1)	22	6	0	0	0	5.7%
Arcee AI: Trinity Mini	14	6	2	1	0	4.7%
GPT-4o, Aug. 6th (temp=0)	10	5	3	0	0	3.6%
Claude 3 Haiku	18	0	0	0	0	3.5%
GPT-4o Mini (temp=1)	4	0	0	0	0	0.8%
GPT-4o Mini (temp=0)	1	0	0	0	0	0.2%
GPT-4o, May 13th (temp=0)	1	0	0	0	0	0.1%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	96	95	84	81	81	87.4%
GPT-5	86	83	80	73	72	78.8%
GPT-5 Mini	86	83	80	69	68	77.4%
Claude Opus 4.6	83	79	78	78	67	77.1%
ByteDance Seed 1.6 Flash	88	81	79	73	62	76.7%
GPT-5.1	81	81	73	69	68	74.4%
Claude Sonnet 4.5	83	75	74	69	68	74.0%
Minimax M2.5	82	79	78	69	54	72.4%
GPT-5.2	83	80	71	65	62	72.2%
ByteDance Seed 1.6	77	74	74	71	57	70.4%
Claude Haiku 4.5	86	70	69	61	51	67.3%
Claude Opus 4	83	81	63	60	43	66.0%
Claude Opus 4.5	71	70	64	63	62	66.0%
Z.AI GLM 4.7 Flash	71	67	56	55	53	60.4%
Z.AI GLM 5	67	66	63	63	43	60.3%
Rocinante 12B	68	68	57	56	49	59.7%
Claude Sonnet 4	76	65	64	64	27	59.1%
Claude 3.7 Sonnet	74	72	59	58	32	59.1%
GPT-5 Nano	59	58	57	56	55	57.0%
WizardLM 2 8x22b	63	59	56	55	51	57.0%
DeepSeek V3.2	68	60	59	58	40	56.9%
MoonshotAI: Kimi K2.5	71	64	57	51	39	56.4%
Mistral Medium 3.1	71	69	64	39	35	55.5%
Z.AI GLM 4.7	55	53	50	49	46	50.8%
Mistral Large 3	70	63	62	47	11	50.7%
GPT-4.1	66	64	41	41	39	50.2%
Z.AI GLM 4.5	62	50	49	46	44	50.0%
Qwen 3.5 397B A17B	71	56	47	35	34	48.9%
Gemini 3 Flash (Preview)	59	52	48	41	38	47.6%
Ministral 3 8B	60	52	52	45	29	47.5%
Writer: Palmyra X5	64	58	38	37	36	46.7%
Ministral 8B	60	49	42	40	38	46.0%
Mistral Small Creative	75	43	37	36	35	45.2%
GPT-4.1 Mini	59	47	42	35	33	43.0%
Z.AI GLM 4.6	57	46	41	41	29	42.7%
Mistral Large	68	49	37	36	20	42.3%
Mistral Large 2	66	66	42	36	0	41.9%
Gemini 3 Pro (Preview)	61	55	43	38	11	41.7%
Cohere Command R+ (Aug. 2024)	63	49	42	40	4	39.6%
Llama 3.1 70B	92	55	17	17	15	39.4%
Gemini 3.1 Pro (Preview)	75	35	33	26	23	38.3%
Llama 3.1 8B	64	57	38	30	0	37.8%
Arcee AI: Trinity Mini	64	53	42	26	0	37.0%
DeepSeek V3.1	57	55	46	27	0	36.7%
o4 Mini	50	42	38	32	20	36.5%
Gemini 2.5 Flash	49	47	43	40	1	36.2%
Hermes 3 70B	57	37	35	33	11	34.6%
Gemma 3 27B	44	41	38	31	18	34.4%
Claude 3.5 Sonnet	52	49	37	19	12	33.8%
GPT-4.1 Nano	49	41	33	18	18	31.9%
Gemini 2.5 Pro	43	40	29	21	20	30.6%
DeepSeek V3 (2025-03-24)	46	38	34	18	17	30.5%
DeepSeek-V2 Chat	46	37	35	24	10	30.5%
Qwen 3.5 Plus (2026-02-15)	46	39	36	19	10	29.8%
Arcee AI: Trinity Large (Preview)	58	46	45	0	0	29.8%
GPT-4o, May 13th (temp=0)	42	40	34	25	5	29.2%
Hermes 3 405B	46	35	28	20	15	28.7%
Claude 3 Haiku	46	42	30	19	6	28.6%
DeepSeek V3 (2024-12-26)	37	36	32	27	0	26.3%
Grok 4.1 Fast	41	31	27	21	10	26.1%
Ministral 3B	41	31	31	26	0	25.8%
Mistral Small 3.2 24B	98	27	0	0	0	25.0%
Gemma 3 12B	50	35	22	14	0	24.1%
Grok 4	32	30	25	22	9	23.7%
Ministral 3 14B	42	39	16	16	4	23.3%
GPT-4o, Aug. 6th (temp=0)	51	30	18	14	0	22.7%
Llama 3.1 Nemotron 70B	58	25	17	13	0	22.6%
Grok 4 Fast	36	33	30	14	0	22.5%
GPT-4o, May 13th (temp=1)	38	35	25	6	3	21.3%
o4 Mini High	51	23	16	9	0	19.8%
Gemini 2.5 Flash Lite	41	10	10	8	5	14.9%
Qwen 2.5 72B	24	15	10	9	1	11.9%
Ministral 3 3B	27	22	8	0	0	11.5%
Stealth: Aurora Alpha	24	14	12	8	0	11.4%
GPT-4o Mini (temp=1)	27	12	0	0	0	7.7%
Gemma 3 4B	13	6	5	4	0	5.6%
Mistral NeMO	20	8	0	0	0	5.6%
Claude 3.5 Haiku	19	2	0	0	0	4.3%
GPT-4o Mini (temp=0)	7	0	0	0	0	1.5%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	100	90	79	74	74	83.5%
GPT-5	84	83	78	77	77	79.8%
GPT-5 Mini	84	82	78	78	69	78.2%
ByteDance Seed 1.6 Flash	84	79	77	72	67	75.7%
ByteDance Seed 1.6	89	75	70	69	64	73.2%
Minimax M2.5	81	76	76	75	58	73.2%
Claude Opus 4.6	82	82	71	68	60	72.5%
GPT-5.1	75	70	68	66	54	66.7%
Qwen 3.5 397B A17B	77	70	67	59	55	65.6%
Claude Opus 4.5	80	72	67	62	46	65.5%
GPT-5.2	69	68	65	63	60	64.9%
Writer: Palmyra X5	74	72	72	63	42	64.7%
Claude Haiku 4.5	78	78	66	57	44	64.6%
Rocinante 12B	82	80	70	62	23	63.3%
Z.AI GLM 4.7	68	68	68	57	51	62.4%
Claude Sonnet 4.5	84	73	56	53	46	62.3%
Mistral Large	77	65	59	55	52	61.4%
Z.AI GLM 5	71	60	60	58	50	59.8%
Claude Opus 4	71	65	57	56	40	57.7%
Mistral Large 3	72	55	54	54	49	56.5%
Mistral Small Creative	66	64	59	47	46	56.3%
MoonshotAI: Kimi K2.5	77	63	50	47	40	55.1%
WizardLM 2 8x22b	68	62	55	53	35	54.8%
GPT-5 Nano	63	57	51	51	49	54.1%
Z.AI GLM 4.5	70	66	58	42	33	53.9%
Ministral 3 14B	78	76	47	36	32	53.8%
Gemini 3 Flash (Preview)	65	59	52	45	41	52.4%
Mistral Large 2	81	60	50	39	31	52.4%
GPT-4.1	72	52	46	45	43	51.5%
DeepSeek V3.2	52	50	49	47	41	47.8%
Gemini 3 Pro (Preview)	72	57	43	42	23	47.6%
DeepSeek V3 (2025-03-24)	68	58	45	35	30	47.2%
Mistral Medium 3.1	73	57	43	39	23	47.0%
Z.AI GLM 4.6	60	59	47	34	34	46.9%
DeepSeek V3.1	64	62	51	29	28	46.9%
Qwen 3.5 Plus (2026-02-15)	73	53	41	37	25	46.0%
DeepSeek V3 (2024-12-26)	64	52	46	38	25	44.9%
Z.AI GLM 4.7 Flash	51	48	39	39	32	41.9%
Claude Sonnet 4	60	54	33	27	18	38.3%
Claude 3.7 Sonnet	62	38	38	27	21	37.1%
Claude 3.5 Sonnet	64	55	28	18	15	36.1%
Ministral 3 8B	58	49	46	19	8	35.8%
Hermes 3 70B	64	44	39	30	0	35.4%
Hermes 3 405B	66	38	36	26	9	34.8%
Grok 4	51	40	30	29	23	34.8%
o4 Mini	52	46	26	26	25	34.7%
Ministral 8B	48	43	38	22	9	32.0%
Cohere Command R+ (Aug. 2024)	64	58	28	10	0	31.9%
o4 Mini High	49	43	25	22	8	29.6%
Gemini 2.5 Pro	44	39	38	17	10	29.5%
DeepSeek-V2 Chat	41	34	33	25	12	29.0%
Llama 3.1 8B	77	63	5	0	0	28.9%
Gemini 3.1 Pro (Preview)	59	43	20	5	0	25.5%
Grok 4.1 Fast	32	29	26	25	13	25.2%
Ministral 3 3B	46	40	32	0	0	23.7%
Grok 4 Fast	42	33	17	16	11	23.7%
Gemma 3 27B	32	29	25	25	1	22.3%
Arcee AI: Trinity Large (Preview)	55	22	20	5	0	20.5%
Stealth: Aurora Alpha	26	22	20	19	14	20.3%
Mistral NeMO	51	23	20	6	1	20.2%
Arcee AI: Trinity Mini	47	22	17	4	2	18.4%
Qwen 2.5 72B	47	21	15	7	0	17.9%
Gemma 3 4B	44	29	14	0	0	17.3%
Claude 3.5 Haiku	70	13	0	0	0	16.6%
Llama 3.1 70B	77	0	0	0	0	15.4%
Gemma 3 12B	28	22	19	6	1	15.1%
GPT-4o, May 13th (temp=0)	38	15	14	0	0	13.4%
Gemini 2.5 Flash	29	23	14	1	0	13.3%
GPT-4o, Aug. 6th (temp=1)	24	22	0	0	0	9.3%
GPT-4.1 Nano	24	10	9	0	0	8.5%
GPT-4.1 Mini	26	10	5	0	0	8.3%
Mistral Small 3.2 24B	22	10	0	0	0	6.5%
Claude 3 Haiku	21	5	0	0	0	5.3%
Gemini 2.5 Flash Lite	14	2	0	0	0	3.1%
GPT-4o, May 13th (temp=1)	10	3	0	0	0	2.6%
Ministral 3B	11	0	0	0	0	2.2%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=1)	0	0	0	0	0	0.0%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%
Llama 3.1 Nemotron 70B	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	90	86	80	76	76	81.7%
GPT-5	85	84	79	75	69	78.6%
GPT-5 Mini	86	79	78	68	67	75.7%
GPT-5.2	77	75	74	73	66	72.9%
ByteDance Seed 1.6 Flash	86	77	72	64	57	71.4%
Claude Opus 4.5	76	76	70	69	65	71.3%
Claude Opus 4.6	82	77	72	67	53	70.2%
Minimax M2.5	84	72	70	67	53	69.1%
Claude Opus 4	82	70	66	59	54	66.1%
GPT-5.1	74	69	69	61	47	63.8%
Z.AI GLM 4.7	75	66	66	58	54	63.8%
Claude Sonnet 4.5	69	66	62	55	53	61.1%
WizardLM 2 8x22b	89	59	59	52	37	59.3%
ByteDance Seed 1.6	62	61	59	57	56	59.0%
Z.AI GLM 5	75	63	54	51	37	56.1%
Claude 3.7 Sonnet	67	65	65	58	24	55.8%
Writer: Palmyra X5	67	61	54	53	37	54.4%
Qwen 3.5 397B A17B	64	63	62	47	34	53.9%
DeepSeek V3.2	66	65	57	48	33	53.8%
Claude Sonnet 4	59	57	56	52	45	53.7%
Mistral Small Creative	64	63	52	46	40	52.9%
Llama 3.1 8B	64	63	54	52	31	52.9%
Z.AI GLM 4.7 Flash	78	65	50	39	32	52.6%
Z.AI GLM 4.5	73	50	49	45	41	51.5%
Z.AI GLM 4.6	71	65	46	46	21	49.7%
Gemini 3 Pro (Preview)	70	64	38	36	34	48.4%
GPT-5 Nano	51	50	48	47	45	48.1%
Claude Haiku 4.5	66	55	46	39	33	47.7%
DeepSeek V3 (2025-03-24)	64	50	45	43	21	44.6%
Mistral Large 2	61	47	39	37	35	43.7%
Gemini 3.1 Pro (Preview)	63	52	37	33	32	43.2%
MoonshotAI: Kimi K2.5	85	46	37	36	10	42.9%
Qwen 3.5 Plus (2026-02-15)	59	58	44	27	25	42.7%
Gemini 3 Flash (Preview)	52	49	46	35	30	42.6%
DeepSeek V3.1	54	43	39	36	33	41.1%
Claude 3.5 Sonnet	73	52	42	24	11	40.4%
Mistral Medium 3.1	55	49	35	33	29	39.8%
Ministral 8B	58	56	37	23	19	38.5%
Arcee AI: Trinity Large (Preview)	65	62	27	21	9	36.7%
Cohere Command R+ (Aug. 2024)	66	43	36	33	0	35.7%
Rocinante 12B	78	50	22	16	0	33.2%
GPT-4.1	40	38	31	25	24	31.5%
Hermes 3 70B	44	42	27	26	14	30.5%
Ministral 3 8B	85	42	22	2	0	30.1%
Mistral Large	59	59	13	10	3	28.8%
Ministral 3 14B	42	38	27	15	10	26.3%
Gemma 3 27B	50	35	31	11	4	26.1%
DeepSeek-V2 Chat	49	28	21	14	5	23.5%
Mistral Large 3	39	35	14	11	8	21.6%
Gemma 3 12B	34	24	23	14	11	21.2%
o4 Mini High	38	33	20	12	0	20.7%
Gemini 2.5 Pro	34	21	20	14	13	20.4%
Ministral 3 3B	50	28	16	9	0	20.3%
DeepSeek V3 (2024-12-26)	40	32	15	12	0	19.9%
GPT-4o, May 13th (temp=0)	43	31	22	0	0	19.3%
Claude 3 Haiku	25	22	22	17	3	17.8%
GPT-4.1 Mini	40	33	15	0	0	17.6%
Ministral 3B	34	26	19	4	0	16.5%
Grok 4	25	20	18	11	2	15.2%
Gemma 3 4B	42	11	9	9	5	15.1%
GPT-4.1 Nano	37	19	18	1	0	14.9%
Arcee AI: Trinity Mini	32	16	15	9	0	14.4%
Llama 3.1 Nemotron 70B	32	17	15	4	4	14.3%
Grok 4 Fast	23	22	13	9	4	14.3%
Claude 3.5 Haiku	54	8	0	0	0	12.4%
Hermes 3 405B	31	15	11	1	0	11.7%
Mistral Small 3.2 24B	30	28	0	0	0	11.6%
Qwen 2.5 72B	30	14	8	0	0	10.5%
Llama 3.1 70B	32	13	5	0	0	10.1%
Grok 4.1 Fast	17	15	14	3	0	9.5%
o4 Mini	25	19	1	0	0	8.9%
Gemini 2.5 Flash Lite	13	3	2	0	0	3.6%
GPT-4o Mini (temp=1)	17	0	0	0	0	3.3%
GPT-4o, May 13th (temp=1)	5	3	3	0	0	2.2%
Mistral NeMO	9	0	0	0	0	1.9%
GPT-4o Mini (temp=0)	5	0	0	0	0	1.0%
Gemini 2.5 Flash	5	0	0	0	0	1.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=1)	0	0	0	0	0	0.0%
GPT-4o, Aug. 6th (temp=0)	0	0	0	0	0	0.0%

AI-ism word frequency

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets