Name drop frequency

Test: Bad Writing Habits

Avg. Score

66.0%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	GPT-4.1 Nano	94.8%	$0.0007	13.3s	81%
2	Gemma 3 4B	88.9%	$0.0002	20.0s	65%
3	Gemini 2.5 Pro	92.7%	$0.036	36.2s	69%
4	Grok 4 Fast	86.6%	$0.0017	24.1s	60%
5	Z.AI GLM 4.6	89.7%	$0.0065	51.5s	59%
6	DeepSeek V3.1	90.6%	$0.0020	1.8m	69%
7	GPT-5 Nano	85.9%	$0.0042	1.4m	63%
8	Stealth: Aurora Alpha	77.7%	$0.0000	9.8s	48%
9	GPT-4o Mini (temp=1)	80.4%	$0.0012	34.8s	48%
10	Gemini 2.5 Flash	80.2%	$0.0052	10.6s	42%
11	Gemini 2.5 Flash Lite	78.5%	$0.0009	9.5s	41%
12	GPT-5 Mini	82.2%	$0.0100	57.4s	47%
13	DeepSeek V3.2	83.5%	$0.0014	1.9m	54%
14	Arcee AI: Trinity Mini	76.0%	$0.0003	9.2s	36%
15	ByteDance Seed 1.6	84.3%	$0.013	2.5m	53%
16	Gemma 3 27B	76.7%	$0.0006	52.6s	35%
17	Claude Sonnet 4.6	81.4%	$0.031	39.3s	37%
18	GPT-4o, Aug. 6th (temp=1)	72.7%	$0.018	24.4s	36%
19	GPT-4.1 Mini	71.9%	$0.0027	19.0s	30%
20	Claude 3.5 Haiku	69.1%	$0.0035	10.8s	32%
21	Gemini 3 Flash (Preview)	68.4%	$0.0078	19.6s	36%
22	Z.AI GLM 5	77.1%	$0.0084	1.2m	37%
23	Grok 4.1 Fast	71.8%	$0.0018	37.8s	32%
24	Minimax M2.5	75.3%	$0.0034	1.3m	35%
25	GPT-4.1	71.7%	$0.018	44.7s	36%
26	Mistral Medium 3.1	67.9%	$0.0048	36.5s	31%
27	Z.AI GLM 4.7 Flash	71.5%	$0.0017	1.2m	33%
28	o4 Mini High	73.6%	$0.025	47.2s	31%
29	Mistral Small Creative	62.1%	$0.0007	9.1s	26%
30	Claude Haiku 4.5	67.0%	$0.011	21.6s	27%
31	Grok 4	80.3%	$0.048	1.7m	43%
32	Gemma 3 12B	68.7%	$0.0004	41.3s	25%
33	Qwen 3.5 Plus (2026-02-15)	64.3%	$0.0060	31.5s	28%
34	Ministral 3 14B	59.1%	$0.0007	11.7s	27%
35	Claude Sonnet 4	69.2%	$0.032	43.7s	34%
36	o4 Mini	67.0%	$0.015	25.7s	26%
37	Mistral Large 3	61.2%	$0.0033	30.3s	28%
38	Ministral 3 8B	59.7%	$0.0008	19.6s	26%
39	DeepSeek V3 (2024-12-26)	64.7%	$0.0021	54.6s	26%
40	Writer: Palmyra X5	64.7%	$0.011	22.0s	21%
41	Ministral 8B	57.3%	$0.0004	10.4s	22%
42	Z.AI GLM 4.7	69.1%	$0.010	1.4m	29%
43	GPT-4o Mini (temp=0)	61.3%	$0.0012	34.8s	22%
44	Llama 3.1 8B	67.8%	$0.0003	1.3m	24%
45	WizardLM 2 8x22b	69.3%	$0.0026	1.8m	30%
46	ByteDance Seed 1.6 Flash	57.8%	$0.0013	27.3s	24%
47	Mistral NeMO	55.0%	$0.0005	10.1s	22%
48	Rocinante 12B	59.4%	$0.0014	38.4s	24%
49	Cohere Command R+ (Aug. 2024)	64.2%	$0.020	52.5s	28%
50	DeepSeek-V2 Chat	61.4%	$0.0021	53.3s	25%
51	Ministral 3B	53.1%	$0.0001	8.1s	22%
52	Claude Opus 4.6	77.6%	$0.078	1.2m	37%
53	Mistral Large 2	57.5%	$0.013	29.4s	25%
54	Mistral Large	57.1%	$0.014	30.9s	25%
55	Hermes 3 70B	59.7%	$0.0010	1.2m	25%
56	DeepSeek V3 (2025-03-24)	55.6%	$0.0014	39.4s	22%
57	Llama 3.1 Nemotron 70B	53.2%	$0.0038	31.7s	24%
58	GPT-5	80.8%	$0.065	2.8m	45%
59	Claude Sonnet 4.5	62.8%	$0.035	38.1s	23%
60	GPT-4o, Aug. 6th (temp=0)	55.7%	$0.023	22.7s	22%
61	Arcee AI: Trinity Large (Preview)	55.2%	$0.0000	43.6s	19%
62	Claude Opus 4.5	67.7%	$0.070	53.4s	32%
63	Ministral 3 3B	47.8%	$0.0005	11.1s	19%
64	MoonshotAI: Kimi K2.5	73.1%	$0.019	3.2m	36%
65	Llama 3.1 70B	47.7%	$0.0015	29.4s	19%
66	GPT-4o, May 13th (temp=1)	54.0%	$0.033	14.4s	18%
67	GPT-5.1	71.7%	$0.054	1.8m	23%
68	Gemini 3 Pro (Preview)	61.3%	$0.055	54.4s	22%
69	Hermes 3 405B	50.4%	$0.0032	53.2s	15%
70	Qwen 2.5 72B	40.8%	$0.0010	36.7s	21%
71	Claude 3.5 Sonnet	55.0%	$0.048	35.5s	21%
72	Z.AI GLM 4.5	42.0%	$0.0051	42.1s	14%
73	Claude 3 Haiku	30.6%	$0.0025	14.9s	16%
74	Qwen 3.5 397B A17B	57.9%	$0.014	3.0m	22%
75	GPT-4o, May 13th (temp=0)	42.3%	$0.035	14.1s	9%
76	Gemini 3.1 Pro (Preview)	64.3%	$0.107	1.8m	25%
77	Claude 3.7 Sonnet	37.5%	$0.042	46.7s	14%
78	GPT-5.2	28.0%	$0.056	1.5m	6%
79	Mistral Small 3.2 24B	52.8%	$0.0069	5.7m	15%
80	Claude Opus 4	56.3%	$0.209	1.4m	22%
66.02%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-4.1 Nano	100	100	100	98	94	98.3%
Gemma 3 4B	100	100	100	92	83	95.2%
Z.AI GLM 4.7	100	100	97	83	69	89.8%
Grok 4 Fast	99	91	89	86	83	89.7%
Claude Sonnet 4.6	100	86	83	83	83	87.1%
Gemini 3.1 Pro (Preview)	100	100	83	83	67	86.7%
Llama 3.1 8B	100	100	83	80	66	85.8%
DeepSeek V3.2	100	94	83	83	67	85.5%
GPT-4.1 Mini	100	100	80	78	67	85.0%
Grok 4	92	91	85	83	68	83.8%
GPT-5	100	100	83	83	50	83.3%
GPT-5 Nano	100	100	83	67	67	83.3%
Stealth: Aurora Alpha	100	97	83	67	67	82.7%
Gemma 3 12B	100	98	98	67	50	82.5%
Claude 3.5 Haiku	100	94	89	64	49	79.1%
GPT-5.1	100	83	83	78	50	78.9%
Gemini 2.5 Pro	100	100	91	67	33	78.3%
Gemini 3 Pro (Preview)	100	100	83	63	43	77.8%
ByteDance Seed 1.6	83	82	82	71	70	77.6%
DeepSeek V3.1	100	84	78	67	50	75.8%
Ministral 3 3B	93	87	79	69	50	75.7%
Z.AI GLM 4.6	96	94	83	67	33	74.8%
GPT-5 Mini	100	98	67	50	50	73.0%
Arcee AI: Trinity Large (Preview)	100	100	67	56	36	71.7%
Grok 4.1 Fast	82	75	67	67	67	71.6%
Claude Opus 4.6	87	82	76	62	50	71.4%
Z.AI GLM 5	100	100	62	61	33	71.2%
Gemini 3 Flash (Preview)	83	83	71	67	50	70.8%
Ministral 3B	94	93	67	48	46	69.5%
Gemini 2.5 Flash Lite	84	82	67	64	50	69.4%
GPT-4o Mini (temp=1)	84	83	67	62	50	69.0%
Gemini 2.5 Flash	83	81	76	50	50	68.2%
Gemma 3 27B	100	79	67	61	33	68.0%
Claude Haiku 4.5	82	67	65	64	59	67.2%
Llama 3.1 Nemotron 70B	100	67	67	61	39	66.7%
Hermes 3 70B	88	78	50	50	49	63.1%
Ministral 3 8B	90	68	65	50	39	62.5%
Z.AI GLM 4.7 Flash	94	83	58	41	33	61.9%
Mistral Large	83	75	50	50	47	61.0%
Mistral Small 3.2 24B	89	81	69	67	0	61.0%
Ministral 8B	100	67	56	48	33	60.8%
Qwen 3.5 Plus (2026-02-15)	100	78	66	59	0	60.6%
Minimax M2.5	83	77	50	48	42	59.9%
ByteDance Seed 1.6 Flash	67	61	60	56	55	59.7%
MoonshotAI: Kimi K2.5	72	68	64	54	33	58.2%
o4 Mini	63	60	57	55	52	57.5%
WizardLM 2 8x22b	100	67	58	50	0	55.0%
Claude Sonnet 4	67	58	56	50	42	54.5%
Mistral Large 2	67	60	56	55	34	54.5%
Hermes 3 405B	93	90	68	19	0	54.1%
Llama 3.1 70B	82	71	62	50	0	53.2%
Mistral Large 3	67	55	50	49	45	53.2%
Arcee AI: Trinity Mini	72	67	63	37	27	53.1%
DeepSeek-V2 Chat	68	53	51	50	39	52.3%
Mistral Medium 3.1	66	61	60	42	27	51.3%
Mistral NeMO	82	67	59	44	0	50.3%
Claude Opus 4	64	54	50	50	33	50.2%
Rocinante 12B	94	67	50	33	0	48.8%
Cohere Command R+ (Aug. 2024)	100	50	48	46	0	48.8%
Claude Sonnet 4.5	70	68	44	44	14	47.9%
GPT-4o, Aug. 6th (temp=1)	67	59	49	45	18	47.4%
Mistral Small Creative	67	66	38	33	31	47.0%
Qwen 2.5 72B	63	55	50	43	19	46.0%
DeepSeek V3 (2025-03-24)	63	56	42	38	29	45.6%
Writer: Palmyra X5	67	59	53	50	0	45.6%
Ministral 3 14B	67	64	49	27	17	44.9%
GPT-4.1	54	50	50	35	33	44.4%
o4 Mini High	83	51	48	21	17	44.1%
Claude Opus 4.5	67	50	44	33	17	42.4%
DeepSeek V3 (2024-12-26)	65	54	42	22	16	39.6%
GPT-4o Mini (temp=0)	67	56	33	33	0	37.9%
Claude 3.5 Sonnet	45	41	40	32	19	35.4%
Qwen 3.5 397B A17B	60	47	45	17	0	33.7%
GPT-4o, May 13th (temp=0)	72	42	24	17	13	33.5%
GPT-5.2	83	50	33	0	0	33.3%
GPT-4o, May 13th (temp=1)	60	50	35	17	0	32.3%
GPT-4o, Aug. 6th (temp=0)	59	35	34	33	0	32.2%
Z.AI GLM 4.5	50	33	29	28	0	28.1%
Claude 3 Haiku	41	33	29	0	0	20.6%
Claude 3.7 Sonnet	33	20	17	0	0	14.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	99	99.9%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	99	99.8%
Z.AI GLM 5	100	100	100	97	96	98.7%
Grok 4 Fast	100	100	100	96	89	97.0%
DeepSeek V3.2	100	100	100	100	83	96.7%
Gemma 3 4B	100	100	100	100	83	96.7%
Writer: Palmyra X5	100	100	100	100	83	96.7%
Gemini 3 Pro (Preview)	100	100	100	100	83	96.7%
Grok 4.1 Fast	100	100	100	97	84	96.1%
GPT-4.1 Mini	100	100	100	92	83	95.0%
WizardLM 2 8x22b	100	100	100	90	83	94.7%
GPT-5 Mini	100	100	100	100	67	93.3%
o4 Mini High	100	100	100	83	83	93.3%
Claude Opus 4.5	100	100	100	83	83	93.3%
Z.AI GLM 4.7 Flash	100	100	100	83	83	93.3%
Gemini 2.5 Flash	100	100	100	100	67	93.3%
Claude 3.5 Haiku	100	100	100	93	72	93.1%
Mistral Large 3	100	100	94	84	70	89.7%
Mistral Small Creative	100	100	90	83	71	88.8%
Qwen 3.5 397B A17B	100	96	89	86	69	88.0%
Gemini 3 Flash (Preview)	100	90	83	83	78	86.9%
Minimax M2.5	100	100	100	100	33	86.7%
DeepSeek V3 (2024-12-26)	100	97	94	74	67	86.4%
Ministral 3 3B	100	97	93	85	54	85.7%
Claude Haiku 4.5	100	100	98	95	33	85.3%
Claude Sonnet 4.5	100	100	83	75	67	85.1%
Qwen 3.5 Plus (2026-02-15)	97	95	83	83	67	85.0%
GPT-4.1	100	100	89	83	50	84.5%
DeepSeek V3 (2025-03-24)	100	100	98	67	56	84.2%
Stealth: Aurora Alpha	100	99	83	83	54	84.0%
Mistral Large	100	100	94	76	50	83.9%
Hermes 3 70B	100	100	100	87	28	83.1%
Arcee AI: Trinity Large (Preview)	100	100	100	63	48	82.3%
o4 Mini	100	100	71	70	61	80.5%
Claude Sonnet 4	100	98	93	62	49	80.2%
MoonshotAI: Kimi K2.5	100	93	83	79	46	80.2%
GPT-4o Mini (temp=0)	100	100	83	67	50	80.0%
Mistral Medium 3.1	96	83	79	74	67	79.8%
DeepSeek-V2 Chat	100	83	74	70	67	78.9%
Mistral Large 2	95	91	91	67	50	78.7%
Ministral 3 14B	99	97	78	63	56	78.7%
Ministral 8B	100	92	92	91	17	78.4%
GPT-5.2	100	86	83	83	33	77.2%
Arcee AI: Trinity Mini	100	100	85	83	17	77.0%
Llama 3.1 8B	100	83	83	60	58	76.9%
Ministral 3 8B	82	80	75	70	67	74.7%
ByteDance Seed 1.6 Flash	83	83	76	67	62	74.3%
Mistral NeMO	100	91	72	67	33	72.7%
Ministral 3B	100	98	75	73	17	72.5%
Cohere Command R+ (Aug. 2024)	100	83	71	54	50	71.8%
Mistral Small 3.2 24B	100	95	83	42	37	71.3%
GPT-4o, Aug. 6th (temp=0)	100	84	83	56	17	68.1%
Z.AI GLM 4.5	82	78	71	67	33	66.1%
GPT-4o, May 13th (temp=1)	95	94	70	48	17	64.6%
Claude Opus 4	76	67	67	63	50	64.6%
Claude 3 Haiku	72	68	67	64	51	64.3%
Llama 3.1 70B	94	88	81	58	0	64.1%
Qwen 2.5 72B	75	71	67	56	50	63.7%
Claude 3.5 Sonnet	100	74	63	50	31	63.6%
Llama 3.1 Nemotron 70B	68	68	67	59	46	61.6%
Claude 3.7 Sonnet	83	81	81	31	17	58.8%
GPT-4o, May 13th (temp=0)	100	90	36	33	17	55.2%
Hermes 3 405B	80	67	48	39	22	51.3%
Rocinante 12B	81	41	17	0	0	27.8%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemma 3 12B	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	83	96.7%
GPT-4.1 Nano	100	100	100	100	83	96.7%
Arcee AI: Trinity Mini	100	100	100	93	82	95.1%
DeepSeek V3.1	100	100	100	100	67	93.3%
GPT-4o Mini (temp=0)	100	100	100	83	83	93.3%
Gemma 3 4B	100	100	100	83	83	93.3%
Llama 3.1 8B	100	100	100	83	67	90.0%
Gemini 2.5 Pro	100	100	100	100	32	86.5%
Mistral Large 2	100	100	96	83	50	85.9%
GPT-5 Mini	100	100	100	67	50	83.3%
Claude Opus 4	100	83	83	75	71	82.5%
Gemma 3 27B	100	100	83	67	62	82.4%
Mistral Large	100	100	78	70	58	81.2%
Z.AI GLM 4.7	100	100	100	67	30	79.3%
DeepSeek V3 (2024-12-26)	100	96	89	64	42	78.3%
GPT-4.1 Mini	100	94	69	66	59	77.6%
GPT-4o Mini (temp=1)	100	100	80	67	33	76.0%
Z.AI GLM 5	100	100	100	80	0	76.0%
Gemini 2.5 Flash	100	100	94	83	0	75.4%
Claude 3.5 Sonnet	100	100	96	41	17	70.6%
Hermes 3 70B	100	83	80	48	39	70.2%
Gemini 2.5 Flash Lite	100	100	83	46	17	69.1%
Stealth: Aurora Alpha	91	67	67	61	50	67.1%
Mistral Medium 3.1	90	82	67	63	33	67.0%
Grok 4	100	78	75	71	0	64.9%
Rocinante 12B	82	79	67	50	41	63.9%
Z.AI GLM 4.6	100	100	93	17	0	61.9%
Claude Opus 4.5	100	83	76	35	14	61.7%
GPT-4o, Aug. 6th (temp=1)	98	91	75	41	0	61.1%
Grok 4 Fast	83	83	73	46	16	60.2%
ByteDance Seed 1.6	100	83	65	33	17	59.7%
GPT-5	83	83	67	50	0	56.7%
DeepSeek V3.2	100	100	50	17	17	56.7%
Minimax M2.5	100	83	67	17	16	56.5%
Llama 3.1 Nemotron 70B	92	67	58	33	29	55.6%
DeepSeek-V2 Chat	100	63	57	57	0	55.4%
Z.AI GLM 4.7 Flash	83	67	60	44	18	54.5%
Ministral 3 8B	83	66	50	33	26	51.7%
Gemini 3 Flash (Preview)	67	50	50	50	33	50.0%
Cohere Command R+ (Aug. 2024)	91	77	35	33	12	49.4%
Claude Sonnet 4	100	100	33	0	0	46.7%
Arcee AI: Trinity Large (Preview)	100	83	50	0	0	46.7%
GPT-4o, May 13th (temp=1)	67	67	67	32	0	46.3%
Grok 4.1 Fast	98	74	50	0	0	44.3%
Hermes 3 405B	100	55	33	32	0	44.0%
Gemini 3 Pro (Preview)	83	67	33	33	0	43.3%
MoonshotAI: Kimi K2.5	67	63	33	33	11	41.5%
Ministral 3 14B	72	69	50	0	0	38.3%
GPT-4.1	100	83	0	0	0	36.7%
Qwen 3.5 Plus (2026-02-15)	67	50	33	33	0	36.7%
WizardLM 2 8x22b	100	83	0	0	0	36.7%
Gemini 3.1 Pro (Preview)	67	50	25	17	17	35.0%
GPT-5.1	100	67	0	0	0	33.3%
o4 Mini	100	65	0	0	0	33.0%
Mistral Large 3	67	67	17	14	0	32.8%
o4 Mini High	100	58	0	0	0	31.5%
Writer: Palmyra X5	90	48	17	0	0	30.9%
DeepSeek V3 (2025-03-24)	57	50	43	3	0	30.5%
Claude 3 Haiku	45	39	33	28	0	29.0%
Claude 3.7 Sonnet	54	50	33	0	0	27.4%
Ministral 8B	98	17	17	0	0	26.3%
Ministral 3B	67	60	0	0	0	25.2%
Mistral NeMO	35	33	33	17	0	23.6%
Mistral Small 3.2 24B	57	33	22	6	0	23.5%
Claude Haiku 4.5	100	17	0	0	0	23.3%
Claude Opus 4.6	67	33	11	0	0	22.3%
Llama 3.1 70B	33	25	24	17	0	19.9%
Ministral 3 3B	49	45	0	0	0	18.7%
Mistral Small Creative	50	17	17	7	0	18.0%
ByteDance Seed 1.6 Flash	83	6	0	0	0	17.9%
Claude 3.5 Haiku	60	22	0	0	0	16.4%
Qwen 3.5 397B A17B	33	31	17	0	0	16.3%
Claude Sonnet 4.5	67	13	0	0	0	15.9%
Claude Sonnet 4.6	33	17	0	0	0	10.0%
GPT-4o, Aug. 6th (temp=0)	33	17	0	0	0	10.0%
Qwen 2.5 72B	33	17	0	0	0	10.0%
GPT-4o, May 13th (temp=0)	41	0	0	0	0	8.3%
Z.AI GLM 4.5	23	0	0	0	0	4.5%
GPT-5.2	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	100	100	100	86	80	93.1%
Z.AI GLM 4.6	100	100	93	77	77	89.3%
Gemma 3 27B	100	100	91	91	59	88.1%
Claude Opus 4.6	100	100	93	83	64	88.0%
GPT-4.1 Nano	100	92	85	83	80	87.8%
Gemma 3 4B	100	100	100	83	50	86.7%
Gemini 2.5 Pro	86	85	83	83	79	83.6%
Arcee AI: Trinity Large (Preview)	100	100	83	67	65	82.9%
GPT-5 Nano	100	100	83	67	50	80.0%
Mistral Large 3	90	83	79	67	67	76.9%
ByteDance Seed 1.6	98	83	67	67	63	75.7%
DeepSeek V3.1	91	86	83	55	48	72.6%
GPT-5	96	83	83	67	33	72.6%
GPT-4o Mini (temp=1)	89	76	73	63	50	70.4%
Gemini 2.5 Flash Lite	83	79	79	73	34	69.7%
GPT-5.1	83	83	82	50	50	69.7%
GPT-4.1	85	72	67	66	50	67.8%
Z.AI GLM 5	97	67	62	61	50	67.2%
Mistral Small Creative	83	83	79	50	37	66.7%
DeepSeek V3.2	83	74	67	67	33	64.9%
Mistral Large	89	79	50	50	44	62.4%
Grok 4 Fast	83	83	74	33	33	61.5%
Minimax M2.5	83	69	67	50	35	60.9%
Hermes 3 70B	100	67	66	66	0	59.7%
Claude Sonnet 4.5	83	69	52	44	38	57.2%
Gemini 3 Flash (Preview)	63	58	50	50	50	54.2%
Qwen 3.5 397B A17B	78	61	60	45	17	52.2%
Z.AI GLM 4.7	78	66	50	33	33	52.2%
Stealth: Aurora Alpha	89	67	46	43	17	52.1%
Mistral Medium 3.1	67	67	63	59	0	51.1%
MoonshotAI: Kimi K2.5	86	64	56	33	13	50.4%
Qwen 3.5 Plus (2026-02-15)	74	51	50	50	17	48.5%
WizardLM 2 8x22b	68	52	50	36	33	48.0%
Arcee AI: Trinity Mini	76	67	50	45	0	47.5%
Gemini 3.1 Pro (Preview)	83	50	50	33	20	47.4%
Gemma 3 12B	74	67	50	37	9	47.2%
Claude Haiku 4.5	67	67	50	31	18	46.5%
Claude Opus 4.5	67	56	45	44	19	46.3%
Writer: Palmyra X5	83	70	63	14	0	45.9%
ByteDance Seed 1.6 Flash	55	52	52	50	12	44.2%
Ministral 3 14B	71	50	42	34	19	43.1%
DeepSeek V3 (2025-03-24)	52	49	46	38	29	43.1%
GPT-4o, Aug. 6th (temp=1)	61	52	35	33	26	41.3%
Gemini 2.5 Flash	70	58	46	29	0	40.5%
Rocinante 12B	65	54	48	33	0	40.2%
Gemini 3 Pro (Preview)	48	46	46	44	17	40.2%
Mistral Large 2	77	50	40	33	0	40.0%
Ministral 8B	71	54	43	26	0	38.7%
Claude 3.5 Haiku	80	55	38	16	0	37.9%
Z.AI GLM 4.7 Flash	59	50	50	17	9	36.9%
GPT-5 Mini	67	50	33	17	17	36.7%
o4 Mini High	50	47	34	33	18	36.4%
Claude Sonnet 4	96	39	20	10	4	33.9%
Cohere Command R+ (Aug. 2024)	77	39	33	18	0	33.5%
Ministral 3 8B	57	56	37	17	0	33.3%
Grok 4.1 Fast	72	33	26	17	17	32.8%
Claude Opus 4	57	33	30	26	17	32.8%
Hermes 3 405B	100	42	6	0	0	29.7%
Grok 4	52	41	38	12	5	29.5%
Mistral NeMO	49	41	17	17	0	24.7%
Llama 3.1 8B	67	50	0	0	0	23.3%
Qwen 2.5 72B	46	38	16	16	0	23.0%
Ministral 3 3B	35	33	23	21	0	22.3%
GPT-4o, May 13th (temp=1)	50	33	22	0	0	21.1%
Ministral 3B	67	12	8	6	2	18.9%
Claude 3.7 Sonnet	76	17	0	0	0	18.6%
Claude 3.5 Sonnet	39	38	1	0	0	15.7%
DeepSeek V3 (2024-12-26)	56	0	0	0	0	11.2%
DeepSeek-V2 Chat	40	14	0	0	0	10.6%
GPT-4.1 Mini	32	17	0	0	0	9.7%
o4 Mini	17	17	13	0	0	9.3%
GPT-4o Mini (temp=0)	43	0	0	0	0	8.7%
GPT-4o, Aug. 6th (temp=0)	27	16	0	0	0	8.6%
Llama 3.1 70B	34	3	0	0	0	7.5%
GPT-5.2	17	17	0	0	0	6.7%
Z.AI GLM 4.5	17	0	0	0	0	3.3%
Mistral Small 3.2 24B	17	0	0	0	0	3.3%
Llama 3.1 Nemotron 70B	9	4	0	0	0	2.5%
GPT-4o, May 13th (temp=0)	0	0	0	0	0	0.0%
Claude 3 Haiku	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 2.5 Pro	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Gemma 3 12B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	95	99.0%
Gemini 3 Flash (Preview)	100	100	100	100	94	98.7%
Gemini 3.1 Pro (Preview)	100	100	100	100	91	98.3%
GPT-4o Mini (temp=1)	100	100	100	100	84	96.8%
o4 Mini High	100	100	100	100	83	96.7%
Gemma 3 4B	100	100	100	100	83	96.7%
Z.AI GLM 5	100	100	100	100	83	96.7%
GPT-4o Mini (temp=0)	100	100	100	100	80	96.1%
MoonshotAI: Kimi K2.5	100	100	100	100	78	95.6%
Claude Opus 4.5	100	100	96	94	83	94.6%
GPT-5 Mini	100	100	100	83	83	93.3%
GPT-5	100	100	100	100	67	93.3%
Minimax M2.5	100	100	100	100	67	93.3%
Claude Sonnet 4.6	100	100	100	83	83	93.3%
Hermes 3 405B	100	100	100	100	66	93.2%
GPT-4.1	100	100	97	90	79	93.1%
Mistral Medium 3.1	100	100	99	83	76	91.8%
Z.AI GLM 4.7	100	100	94	90	67	90.2%
Gemini 3 Pro (Preview)	100	100	88	83	78	89.8%
DeepSeek V3 (2024-12-26)	100	100	100	78	68	89.1%
Z.AI GLM 4.7 Flash	100	100	100	91	50	88.3%
Arcee AI: Trinity Mini	97	96	92	89	67	88.0%
Claude Opus 4.6	100	100	83	83	67	86.7%
GPT-5.1	100	100	83	83	67	86.7%
Grok 4	100	100	100	100	33	86.7%
Z.AI GLM 4.6	100	100	83	83	67	86.7%
Claude 3.5 Haiku	100	100	92	73	68	86.5%
ByteDance Seed 1.6	100	95	83	81	69	85.6%
Writer: Palmyra X5	100	100	92	92	43	85.4%
Claude Sonnet 4	100	100	96	67	59	84.4%
Gemini 2.5 Flash Lite	100	100	100	83	33	83.3%
GPT-4o, Aug. 6th (temp=0)	100	95	81	81	55	82.3%
GPT-5 Nano	100	100	93	67	50	82.0%
Mistral Large 2	100	100	83	75	45	80.6%
ByteDance Seed 1.6 Flash	100	83	83	67	67	80.0%
Claude Haiku 4.5	100	100	97	50	50	79.5%
Ministral 3B	100	100	83	64	42	77.8%
o4 Mini	100	98	87	61	38	77.0%
GPT-4o, Aug. 6th (temp=1)	95	83	83	67	42	74.1%
GPT-4.1 Mini	83	81	81	79	33	71.7%
WizardLM 2 8x22b	100	83	83	67	17	70.0%
Claude 3.5 Sonnet	100	99	55	54	34	68.2%
Llama 3.1 8B	100	100	83	57	0	68.1%
GPT-4o, May 13th (temp=1)	100	92	83	38	11	64.9%
Rocinante 12B	100	100	100	14	0	62.9%
Cohere Command R+ (Aug. 2024)	100	100	80	33	0	62.7%
Qwen 3.5 397B A17B	100	75	67	50	17	61.6%
Stealth: Aurora Alpha	67	67	63	60	50	61.3%
Claude 3.7 Sonnet	83	75	67	65	15	60.9%
DeepSeek-V2 Chat	100	100	100	0	0	60.0%
Llama 3.1 Nemotron 70B	100	67	63	33	33	59.5%
Grok 4.1 Fast	93	83	50	50	17	58.6%
Claude Sonnet 4.5	93	87	67	44	0	58.4%
Qwen 3.5 Plus (2026-02-15)	91	83	50	49	0	54.7%
Ministral 3 14B	95	89	55	17	17	54.4%
Ministral 3 3B	100	100	35	33	0	53.7%
Claude Opus 4	100	100	67	0	0	53.3%
Hermes 3 70B	81	70	67	47	0	52.9%
Mistral Large	100	83	35	32	0	49.9%
Ministral 3 8B	88	84	77	0	0	49.8%
Mistral Small 3.2 24B	83	67	33	30	0	42.6%
Mistral NeMO	90	50	33	17	17	41.3%
Llama 3.1 70B	50	49	45	42	19	41.2%
Mistral Large 3	100	56	33	12	0	40.4%
DeepSeek V3 (2025-03-24)	63	48	46	41	0	39.7%
Claude 3 Haiku	75	59	33	23	0	38.0%
Qwen 2.5 72B	55	49	33	33	11	36.3%
Ministral 8B	99	67	0	0	0	33.2%
GPT-4o, May 13th (temp=0)	66	33	27	17	11	30.8%
Arcee AI: Trinity Large (Preview)	90	40	17	0	0	29.3%
Mistral Small Creative	64	62	17	0	0	28.6%
Z.AI GLM 4.5	67	38	17	0	0	24.2%
GPT-5.2	17	17	11	0	0	8.9%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	92	98.4%
Grok 4 Fast	100	100	100	97	94	98.4%
GPT-4o Mini (temp=1)	100	100	100	99	90	97.9%
Z.AI GLM 4.6	100	100	100	100	83	96.7%
GPT-4.1 Nano	100	100	100	100	83	96.7%
Gemma 3 27B	100	100	100	100	83	96.7%
Grok 4.1 Fast	100	100	95	93	83	94.3%
GPT-5	100	100	100	83	83	93.3%
GPT-5 Mini	100	100	100	83	67	90.0%
GPT-5 Nano	100	100	83	83	83	90.0%
Gemini 2.5 Flash Lite	100	100	100	83	67	90.0%
Writer: Palmyra X5	100	100	100	83	67	90.0%
Claude Opus 4.6	100	100	96	83	67	89.3%
Minimax M2.5	100	99	98	83	62	88.6%
o4 Mini	100	88	86	83	81	87.6%
Mistral Medium 3.1	100	100	100	80	50	86.0%
Claude Opus 4.5	93	86	83	83	76	84.5%
Z.AI GLM 4.7 Flash	100	100	83	76	63	84.4%
MoonshotAI: Kimi K2.5	100	100	83	79	50	82.6%
Gemma 3 4B	100	96	83	67	67	82.5%
Claude Haiku 4.5	100	92	89	67	61	81.7%
Mistral NeMO	100	100	89	86	33	81.7%
Rocinante 12B	100	100	98	90	17	80.9%
o4 Mini High	100	100	80	75	50	80.8%
DeepSeek V3.2	100	100	100	83	17	80.0%
Grok 4	100	94	88	83	33	79.6%
Z.AI GLM 5	100	100	91	83	17	78.1%
Hermes 3 70B	100	100	100	86	3	77.9%
ByteDance Seed 1.6	83	83	78	76	67	77.4%
ByteDance Seed 1.6 Flash	92	83	72	67	67	76.1%
Mistral Small Creative	100	100	79	50	50	75.9%
GPT-4.1 Mini	88	83	81	67	56	75.1%
GPT-4o, Aug. 6th (temp=1)	100	80	78	67	50	75.0%
Gemini 3 Pro (Preview)	100	100	67	56	50	74.6%
Claude Sonnet 4.5	100	83	80	56	50	73.9%
GPT-4o, May 13th (temp=1)	97	79	77	67	49	73.8%
DeepSeek-V2 Chat	88	83	80	55	52	71.7%
GPT-4o Mini (temp=0)	100	88	58	55	55	71.5%
Claude Sonnet 4	95	88	83	59	25	70.1%
Gemma 3 12B	100	67	67	67	50	70.0%
Claude 3.5 Haiku	100	66	63	63	58	70.0%
WizardLM 2 8x22b	100	100	67	60	17	68.7%
Stealth: Aurora Alpha	100	72	61	60	50	68.6%
Mistral Large 3	98	86	76	67	17	68.6%
DeepSeek V3 (2024-12-26)	100	73	68	65	33	67.9%
Ministral 3 14B	85	67	67	58	53	66.0%
Cohere Command R+ (Aug. 2024)	100	77	67	50	33	65.5%
Mistral Large 2	67	67	67	65	60	65.0%
Mistral Small 3.2 24B	100	100	67	39	17	64.5%
Arcee AI: Trinity Large (Preview)	100	100	52	50	17	63.7%
Mistral Large	83	77	50	50	50	62.0%
Gemini 3 Flash (Preview)	73	67	67	50	50	61.2%
Z.AI GLM 4.5	67	66	61	56	50	60.1%
Z.AI GLM 4.7	100	83	83	33	0	60.0%
Gemini 3.1 Pro (Preview)	100	50	50	50	50	60.0%
Ministral 8B	67	66	65	50	44	58.5%
Arcee AI: Trinity Mini	100	72	50	47	20	57.9%
Qwen 3.5 Plus (2026-02-15)	76	67	67	50	17	55.3%
Ministral 3 3B	81	71	54	38	31	54.9%
DeepSeek V3 (2025-03-24)	96	66	60	39	7	53.5%
Ministral 3B	76	73	54	34	21	51.7%
GPT-4.1	71	57	50	50	17	49.0%
Qwen 3.5 397B A17B	94	67	33	30	17	48.1%
GPT-4o, Aug. 6th (temp=0)	83	65	56	33	0	47.4%
Llama 3.1 8B	100	67	50	15	0	46.2%
Qwen 2.5 72B	67	62	52	33	0	42.8%
Ministral 3 8B	67	67	37	33	0	40.7%
Claude 3.5 Sonnet	74	63	31	24	0	38.4%
GPT-4o, May 13th (temp=0)	93	46	19	17	0	35.0%
Hermes 3 405B	56	35	33	31	18	34.8%
Llama 3.1 Nemotron 70B	49	47	39	31	0	33.3%
Claude Opus 4	67	33	17	17	0	26.8%
Claude 3.7 Sonnet	48	31	17	17	0	22.4%
Llama 3.1 70B	47	40	7	1	0	19.0%
Claude 3 Haiku	23	22	17	4	0	13.2%
GPT-5.2	33	0	0	0	0	6.7%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5 Mini	100	100	100	100	93	98.6%
Stealth: Aurora Alpha	100	100	100	95	94	97.9%
Z.AI GLM 4.6	100	100	100	100	83	96.7%
Gemma 3 4B	100	100	100	100	83	96.7%
GPT-4.1 Nano	100	100	100	88	83	94.0%
Arcee AI: Trinity Mini	100	100	91	83	77	90.2%
GPT-4.1 Mini	100	89	89	87	81	89.3%
ByteDance Seed 1.6	99	91	89	80	77	87.1%
Llama 3.1 Nemotron 70B	100	100	91	75	64	86.0%
WizardLM 2 8x22b	95	92	81	80	76	84.8%
GPT-5	100	83	83	83	67	83.3%
Llama 3.1 8B	100	100	100	64	49	82.7%
Grok 4	91	89	83	83	67	82.6%
Llama 3.1 70B	100	93	80	76	59	81.7%
Grok 4.1 Fast	94	87	83	73	69	81.3%
GPT-4o Mini (temp=1)	100	83	76	76	67	80.3%
GPT-5.1	100	100	83	67	50	80.0%
Claude Sonnet 4	100	83	81	78	55	79.7%
Gemini 2.5 Flash	100	83	83	67	65	79.6%
Gemini 2.5 Pro	100	97	93	74	33	79.4%
DeepSeek V3.1	96	83	83	67	67	79.2%
Ministral 8B	86	85	83	69	67	78.0%
Grok 4 Fast	84	83	76	76	71	77.9%
o4 Mini High	87	83	83	75	60	77.6%
GPT-4o, Aug. 6th (temp=0)	87	86	81	67	66	77.4%
Qwen 2.5 72B	91	79	71	67	67	74.9%
DeepSeek V3.2	100	91	83	83	17	74.8%
Claude 3.5 Haiku	96	85	74	62	57	74.6%
GPT-4.1	91	83	73	72	50	73.8%
Claude Opus 4.6	100	89	67	62	50	73.5%
Qwen 3.5 397B A17B	83	76	73	67	65	72.7%
Claude Sonnet 4.6	100	83	73	65	33	70.9%
GPT-4o, Aug. 6th (temp=1)	78	76	68	67	59	69.5%
Minimax M2.5	83	80	67	67	50	69.4%
Gemini 3 Flash (Preview)	81	72	67	62	61	68.7%
Mistral Small 3.2 24B	87	83	67	67	39	68.5%
Cohere Command R+ (Aug. 2024)	89	84	83	50	33	68.0%
Gemma 3 12B	80	74	67	64	50	67.0%
Claude Haiku 4.5	83	79	77	48	45	66.5%
o4 Mini	73	70	66	63	50	64.3%
Ministral 3B	86	64	63	50	44	61.3%
Ministral 3 3B	83	65	59	58	42	61.3%
Gemini 3 Pro (Preview)	78	67	61	50	50	61.1%
Z.AI GLM 4.7	67	67	65	57	50	61.0%
Mistral Medium 3.1	84	71	50	50	50	60.9%
ByteDance Seed 1.6 Flash	83	75	67	41	38	60.7%
Rocinante 12B	99	80	75	49	0	60.5%
DeepSeek V3 (2024-12-26)	82	67	61	54	38	60.4%
DeepSeek-V2 Chat	73	68	58	55	46	60.0%
GPT-5 Nano	83	83	67	50	17	60.0%
Claude Sonnet 4.5	79	67	58	50	44	59.3%
Gemini 3.1 Pro (Preview)	83	67	63	50	33	59.3%
GPT-4o, May 13th (temp=1)	87	66	57	50	33	58.8%
Mistral NeMO	80	68	55	50	38	58.2%
GPT-4o, May 13th (temp=0)	84	60	59	55	33	58.0%
Z.AI GLM 5	83	73	67	33	33	58.0%
Gemma 3 27B	77	69	63	47	32	57.6%
Arcee AI: Trinity Large (Preview)	87	67	66	50	17	57.4%
Hermes 3 70B	81	57	53	50	43	56.7%
Gemini 2.5 Flash Lite	100	67	67	33	17	56.7%
Ministral 3 8B	83	77	62	52	0	55.0%
Z.AI GLM 4.7 Flash	67	53	50	50	50	53.9%
DeepSeek V3 (2025-03-24)	74	59	57	53	25	53.7%
Claude 3.5 Sonnet	85	73	44	33	28	52.8%
Mistral Large 2	74	60	56	50	23	52.4%
Qwen 3.5 Plus (2026-02-15)	77	67	51	33	33	52.3%
GPT-4o Mini (temp=0)	66	52	50	49	33	50.1%
Claude Opus 4.5	83	50	50	33	33	50.0%
MoonshotAI: Kimi K2.5	67	50	50	43	33	48.6%
Ministral 3 14B	78	57	50	36	17	47.4%
Hermes 3 405B	83	76	60	17	0	47.2%
Claude Opus 4	62	58	55	33	28	47.2%
Writer: Palmyra X5	60	50	50	33	33	45.4%
Mistral Large	73	50	33	25	23	40.9%
Mistral Large 3	62	52	45	42	0	40.2%
Z.AI GLM 4.5	54	50	42	28	26	39.9%
Mistral Small Creative	56	50	33	33	17	37.8%
Claude 3.7 Sonnet	58	50	30	27	0	33.1%
Claude 3 Haiku	56	27	17	0	0	19.8%
GPT-5.2	33	17	0	0	0	10.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Llama 3.1 8B	100	100	100	100	100	99.9%
Claude Opus 4.5	100	100	100	100	99	99.9%
Grok 4 Fast	100	100	100	100	99	99.8%
Gemma 3 12B	100	100	100	100	98	99.6%
o4 Mini High	100	100	100	100	97	99.3%
Gemini 3 Flash (Preview)	100	100	100	100	93	98.7%
Mistral Small 3.2 24B	100	100	100	100	92	98.5%
Claude Sonnet 4.5	100	100	99	96	94	97.8%
MoonshotAI: Kimi K2.5	100	100	100	99	89	97.6%
Gemini 3.1 Pro (Preview)	100	100	100	94	93	97.3%
Grok 4.1 Fast	100	100	100	95	88	96.7%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	83	96.7%
GPT-5 Nano	100	100	100	100	83	96.7%
Writer: Palmyra X5	100	100	100	100	83	96.7%
GPT-4o Mini (temp=0)	100	100	100	100	83	96.7%
Arcee AI: Trinity Mini	100	100	100	100	83	96.7%
Mistral Large 3	100	100	100	98	84	96.4%
DeepSeek V3 (2024-12-26)	100	100	99	96	86	96.2%
Gemma 3 27B	100	100	100	98	83	96.2%
GPT-4.1 Mini	100	100	100	100	77	95.5%
o4 Mini	100	100	100	97	80	95.4%
Claude 3.5 Sonnet	100	100	100	91	85	95.1%
GPT-4o, Aug. 6th (temp=1)	100	100	100	89	83	94.5%
GPT-4o, May 13th (temp=1)	100	100	100	100	67	93.3%
GPT-4.1	100	100	100	83	70	90.6%
Gemini 3 Pro (Preview)	100	100	100	100	50	90.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	50	90.0%
Ministral 3 8B	100	93	91	82	79	88.8%
Mistral Large 2	100	99	90	87	67	88.6%
Mistral Large	100	96	85	82	78	88.1%
Ministral 3B	99	95	83	81	80	87.6%
Claude Sonnet 4	100	100	83	83	67	86.7%
Z.AI GLM 4.5	100	95	88	84	67	86.6%
Stealth: Aurora Alpha	100	100	92	67	67	85.1%
Ministral 3 14B	93	91	88	83	67	84.5%
ByteDance Seed 1.6 Flash	100	100	82	67	67	83.1%
WizardLM 2 8x22b	100	83	83	73	73	82.4%
DeepSeek V3 (2025-03-24)	93	89	88	72	64	81.2%
GPT-4o, Aug. 6th (temp=0)	100	85	84	69	67	81.0%
Llama 3.1 Nemotron 70B	100	82	79	74	66	80.3%
Mistral NeMO	100	100	92	71	33	79.3%
DeepSeek-V2 Chat	100	100	85	67	42	78.8%
Mistral Small Creative	100	88	83	67	50	77.6%
GPT-4o, May 13th (temp=0)	100	100	100	50	33	76.7%
Ministral 8B	100	100	82	67	33	76.4%
Arcee AI: Trinity Large (Preview)	98	94	67	58	48	72.9%
Ministral 3 3B	100	86	82	59	33	72.1%
Mistral Medium 3.1	99	81	67	61	50	71.5%
Claude Opus 4	100	100	83	33	33	70.0%
Llama 3.1 70B	100	78	64	59	49	69.8%
Hermes 3 70B	91	90	64	50	49	68.8%
Hermes 3 405B	100	100	56	43	32	66.2%
Rocinante 12B	100	79	66	60	0	61.0%
Qwen 2.5 72B	93	77	67	33	0	54.0%
Claude 3.7 Sonnet	74	55	55	33	33	50.2%
Claude 3 Haiku	74	63	40	36	33	49.2%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	98	99.7%
Grok 4	100	100	100	100	83	96.7%
Stealth: Aurora Alpha	100	100	100	100	83	96.7%
DeepSeek V3.2	100	100	100	100	83	96.7%
GPT-4.1 Nano	100	100	100	98	72	94.1%
DeepSeek V3.1	100	100	100	100	67	93.3%
Gemini 2.5 Flash	100	100	100	83	79	92.4%
Claude 3.5 Haiku	100	98	91	82	76	89.3%
Ministral 3 8B	100	100	98	97	50	89.2%
Ministral 3 14B	100	92	83	81	80	87.2%
Llama 3.1 Nemotron 70B	100	100	88	80	67	87.1%
o4 Mini High	100	100	100	100	33	86.7%
Mistral Small Creative	100	100	100	67	67	86.7%
Grok 4.1 Fast	100	100	100	94	33	85.4%
Z.AI GLM 4.7	100	83	83	83	67	83.3%
Mistral Large 3	100	83	83	82	67	83.0%
Llama 3.1 8B	100	100	73	68	67	81.7%
Mistral Medium 3.1	100	89	83	67	67	81.1%
GPT-4o Mini (temp=1)	100	95	90	83	33	80.2%
Ministral 8B	100	100	95	72	33	80.1%
Claude Sonnet 4	100	83	83	83	50	80.0%
Grok 4 Fast	100	100	83	83	33	80.0%
GPT-4.1	100	100	100	100	0	79.9%
GPT-4.1 Mini	100	100	99	78	17	78.9%
Qwen 3.5 Plus (2026-02-15)	90	83	83	67	67	78.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	83	0	76.7%
GPT-5 Mini	100	100	67	67	33	73.3%
Z.AI GLM 4.7 Flash	100	83	67	67	50	73.3%
Claude Haiku 4.5	100	83	80	67	25	71.0%
ByteDance Seed 1.6	100	100	87	33	33	70.6%
DeepSeek V3 (2024-12-26)	86	83	83	67	33	70.5%
MoonshotAI: Kimi K2.5	100	100	83	50	12	69.0%
Gemini 2.5 Flash Lite	100	91	67	67	17	68.2%
Rocinante 12B	100	83	67	50	38	67.7%
Gemini 3.1 Pro (Preview)	83	83	67	67	33	66.7%
Gemini 3 Flash (Preview)	83	83	67	66	33	66.4%
DeepSeek V3 (2025-03-24)	100	80	67	67	17	66.1%
WizardLM 2 8x22b	100	67	61	50	50	65.5%
Minimax M2.5	100	100	100	17	0	63.3%
Gemma 3 27B	100	83	67	50	17	63.3%
GPT-4o Mini (temp=0)	83	83	80	67	0	62.6%
Cohere Command R+ (Aug. 2024)	78	67	67	52	50	62.5%
DeepSeek-V2 Chat	100	67	67	67	0	60.0%
Claude Sonnet 4.6	100	100	33	33	17	56.7%
Llama 3.1 70B	100	100	67	17	0	56.7%
Hermes 3 405B	100	89	50	42	0	56.3%
Claude 3.5 Sonnet	83	83	76	17	17	55.2%
Claude Opus 4	92	83	83	16	0	54.9%
Writer: Palmyra X5	100	98	67	0	0	52.8%
Ministral 3B	83	67	60	50	0	52.1%
Arcee AI: Trinity Large (Preview)	83	82	33	33	28	52.0%
Claude 3.7 Sonnet	77	67	67	33	0	48.7%
GPT-4o, Aug. 6th (temp=0)	100	91	33	17	0	48.2%
Claude 3 Haiku	89	58	40	32	16	47.2%
ByteDance Seed 1.6 Flash	99	83	50	0	0	46.6%
Qwen 3.5 397B A17B	82	67	33	33	17	46.4%
Z.AI GLM 5	100	48	41	33	0	44.4%
GPT-5	100	50	33	17	0	40.0%
Mistral Large	83	67	50	0	0	40.0%
GPT-4o, May 13th (temp=0)	100	99	0	0	0	39.8%
Claude Opus 4.5	83	67	33	0	0	36.7%
Mistral NeMO	67	50	33	33	0	36.7%
Mistral Large 2	100	67	17	0	0	36.7%
Claude Opus 4.6	100	33	33	17	0	36.5%
Hermes 3 70B	76	33	28	23	12	34.6%
Claude Sonnet 4.5	100	50	6	0	0	31.2%
Gemini 3 Pro (Preview)	67	50	33	0	0	30.0%
Mistral Small 3.2 24B	67	50	29	0	0	29.1%
Qwen 2.5 72B	78	50	17	0	0	28.9%
Ministral 3 3B	78	33	17	11	0	27.8%
Z.AI GLM 4.5	67	33	17	17	0	26.7%
GPT-4o, May 13th (temp=1)	67	33	12	0	0	22.5%
GPT-5.2	50	0	0	0	0	10.0%
Gemma 3 12B	17	16	0	0	0	6.5%
GPT-5.1	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-4.1 Nano	100	100	100	86	83	93.7%
Grok 4 Fast	99	97	83	83	67	85.8%
GPT-5 Nano	100	100	83	67	67	83.3%
Minimax M2.5	100	93	83	83	50	81.8%
Stealth: Aurora Alpha	100	85	79	77	67	81.6%
ByteDance Seed 1.6	83	83	83	80	50	76.0%
Claude Sonnet 4.5	82	79	72	67	61	72.0%
Hermes 3 70B	100	83	62	55	41	68.3%
Gemini 2.5 Pro	74	73	67	57	50	64.1%
Mistral Medium 3.1	100	99	67	50	0	63.1%
GPT-5 Mini	83	76	67	50	33	61.9%
Mistral NeMO	100	81	49	41	33	60.8%
Rocinante 12B	78	78	66	64	10	59.2%
Mistral Large 3	81	71	67	50	17	57.0%
DeepSeek V3.1	75	56	50	50	50	56.2%
Z.AI GLM 5	94	84	50	33	17	55.5%
Qwen 3.5 Plus (2026-02-15)	83	77	50	35	30	55.1%
GPT-4.1	94	81	67	33	0	55.1%
Gemma 3 4B	72	67	61	52	17	53.6%
MoonshotAI: Kimi K2.5	66	58	52	50	40	53.1%
Z.AI GLM 4.6	74	64	50	45	31	52.7%
Claude Sonnet 4	83	66	48	33	29	51.9%
GPT-4o Mini (temp=1)	81	77	50	33	17	51.5%
WizardLM 2 8x22b	79	72	64	36	0	50.2%
GPT-5	100	83	33	33	0	50.0%
Claude Sonnet 4.6	83	67	50	50	0	50.0%
Gemma 3 27B	65	63	50	33	33	49.0%
Claude Opus 4.6	100	61	50	33	0	48.9%
ByteDance Seed 1.6 Flash	79	68	33	33	30	48.9%
DeepSeek V3.2	74	50	50	33	33	48.1%
GPT-4o, Aug. 6th (temp=1)	83	73	50	17	17	48.0%
Mistral Large 2	75	74	47	38	0	46.9%
Mistral Small 3.2 24B	67	67	51	50	0	46.9%
Gemma 3 12B	100	67	67	0	0	46.7%
Grok 4	80	67	33	33	17	45.9%
Gemini 2.5 Flash Lite	62	50	50	33	33	45.6%
Grok 4.1 Fast	83	63	44	33	0	44.7%
GPT-4o, May 13th (temp=1)	61	49	41	33	33	43.7%
Ministral 3 14B	67	50	33	33	33	43.3%
GPT-5.1	83	50	50	33	0	43.3%
GPT-4o Mini (temp=0)	68	51	50	48	0	43.3%
Ministral 8B	75	59	33	31	17	43.0%
Arcee AI: Trinity Mini	67	57	55	28	0	41.2%
Cohere Command R+ (Aug. 2024)	67	56	48	33	0	40.7%
Claude 3.5 Haiku	60	47	34	22	18	36.3%
Mistral Small Creative	51	50	33	29	17	36.1%
Gemini 3 Flash (Preview)	67	33	33	28	17	35.5%
Llama 3.1 8B	59	50	33	28	1	34.4%
Qwen 3.5 397B A17B	72	39	28	24	0	32.8%
DeepSeek V3 (2025-03-24)	56	52	33	21	0	32.5%
Mistral Large	65	50	30	17	0	32.3%
Ministral 3 3B	50	43	38	26	0	31.5%
o4 Mini	53	49	22	20	13	31.3%
Claude Opus 4.5	52	50	33	17	0	30.5%
o4 Mini High	50	50	30	17	0	29.4%
Ministral 3B	48	45	33	16	0	28.6%
Gemini 3.1 Pro (Preview)	78	63	0	0	0	28.2%
GPT-4o, Aug. 6th (temp=0)	49	30	23	17	10	25.8%
DeepSeek-V2 Chat	42	32	29	17	10	25.7%
GPT-4.1 Mini	50	46	17	10	0	24.6%
Claude 3.5 Sonnet	60	33	27	0	0	23.8%
GPT-5.2	33	33	17	17	17	23.3%
Claude Haiku 4.5	39	30	24	22	0	23.1%
Arcee AI: Trinity Large (Preview)	83	27	4	0	0	22.9%
Ministral 3 8B	58	47	8	0	0	22.5%
Gemini 2.5 Flash	33	33	17	15	0	19.6%
Claude Opus 4	76	20	1	0	0	19.4%
Z.AI GLM 4.7 Flash	33	28	17	17	0	19.0%
Hermes 3 405B	49	33	7	2	0	18.3%
Qwen 2.5 72B	27	25	19	17	0	17.4%
Z.AI GLM 4.7	50	33	0	0	0	16.7%
Writer: Palmyra X5	50	17	17	0	0	16.7%
Llama 3.1 Nemotron 70B	54	18	7	0	0	15.8%
Llama 3.1 70B	48	24	4	0	0	15.3%
DeepSeek V3 (2024-12-26)	33	17	17	3	0	14.0%
GPT-4o, May 13th (temp=0)	19	19	0	0	0	7.6%
Claude 3 Haiku	16	11	0	0	0	5.4%
Gemini 3 Pro (Preview)	17	8	0	0	0	4.9%
Claude 3.7 Sonnet	17	0	0	0	0	3.3%
Z.AI GLM 4.5	4	0	0	0	0	0.7%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5 Mini	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	99	99.8%
GPT-4o Mini (temp=1)	100	100	100	100	97	99.4%
GPT-4.1 Mini	100	100	100	100	96	99.2%
Claude Opus 4	100	100	100	98	95	98.5%
Claude Sonnet 4.6	100	100	100	100	92	98.4%
MoonshotAI: Kimi K2.5	100	100	100	100	83	96.7%
Z.AI GLM 4.6	100	100	100	100	83	96.7%
GPT-4.1	100	100	100	100	83	96.7%
DeepSeek V3.2	100	100	100	100	83	96.7%
DeepSeek V3.1	100	100	100	100	83	96.7%
Z.AI GLM 5	100	100	100	100	83	96.7%
Grok 4 Fast	100	100	100	100	83	96.7%
GPT-5 Nano	100	100	100	99	83	96.5%
GPT-4o, Aug. 6th (temp=1)	100	100	100	95	75	93.9%
Qwen 3.5 Plus (2026-02-15)	100	100	96	90	83	93.8%
Claude Opus 4.6	100	100	100	100	67	93.3%
GPT-5	100	100	100	100	67	93.3%
Claude Sonnet 4.5	100	100	100	100	67	93.3%
Stealth: Aurora Alpha	100	100	100	100	67	93.3%
ByteDance Seed 1.6 Flash	100	100	100	100	67	93.3%
Llama 3.1 8B	100	100	100	83	83	93.3%
Arcee AI: Trinity Mini	100	100	100	90	76	93.2%
Claude Opus 4.5	100	100	91	91	83	93.0%
Claude 3.5 Haiku	100	100	100	89	74	92.6%
Claude 3.5 Sonnet	100	94	91	87	83	91.3%
GPT-5.1	100	100	83	83	83	90.0%
Gemini 2.5 Flash Lite	100	100	100	83	67	90.0%
Gemma 3 12B	100	100	100	97	50	89.3%
Claude Sonnet 4	100	100	100	83	63	89.2%
GPT-4o Mini (temp=0)	100	100	100	95	50	88.9%
Claude Haiku 4.5	100	100	83	82	77	88.4%
Z.AI GLM 4.7	100	83	83	83	83	86.7%
ByteDance Seed 1.6	100	100	83	83	67	86.7%
Gemini 2.5 Flash	100	100	83	83	67	86.7%
Mistral Medium 3.1	100	100	96	67	67	85.8%
o4 Mini	100	93	83	83	67	85.2%
Gemini 3.1 Pro (Preview)	83	83	83	83	83	83.3%
Qwen 3.5 397B A17B	100	95	83	67	67	82.4%
Gemini 3 Pro (Preview)	100	93	83	67	67	81.9%
GPT-4o, Aug. 6th (temp=0)	100	94	83	75	52	81.0%
Z.AI GLM 4.5	92	83	83	79	67	80.6%
Ministral 3 14B	97	97	82	78	50	80.6%
Gemini 3 Flash (Preview)	85	83	83	83	67	80.3%
DeepSeek V3 (2024-12-26)	100	94	83	67	50	78.8%
Rocinante 12B	100	90	86	76	33	77.1%
WizardLM 2 8x22b	100	83	75	69	57	76.8%
Hermes 3 405B	100	100	90	83	0	74.7%
GPT-4o, May 13th (temp=0)	96	92	83	83	17	74.3%
Ministral 3B	97	85	67	67	44	71.7%
Cohere Command R+ (Aug. 2024)	91	83	83	66	33	71.4%
Mistral Large 3	100	95	70	52	40	71.3%
DeepSeek-V2 Chat	100	100	83	67	0	70.0%
Claude 3.7 Sonnet	100	71	67	53	50	68.2%
Arcee AI: Trinity Large (Preview)	100	100	100	33	0	66.7%
Llama 3.1 70B	100	79	64	45	33	64.4%
GPT-4o, May 13th (temp=1)	100	83	79	50	0	62.5%
Ministral 3 3B	79	74	70	47	38	61.5%
DeepSeek V3 (2025-03-24)	95	94	67	50	0	61.3%
Hermes 3 70B	100	83	67	32	22	60.8%
Llama 3.1 Nemotron 70B	81	77	69	42	33	60.6%
Gemma 3 27B	100	100	100	0	0	60.0%
Mistral Small 3.2 24B	77	67	50	49	41	56.7%
Qwen 2.5 72B	81	67	65	50	0	52.5%
Mistral NeMO	85	50	50	42	17	48.8%
Ministral 3 8B	98	62	59	17	0	47.2%
Claude 3 Haiku	56	54	45	36	26	43.2%
GPT-5.2	72	50	33	33	17	41.1%
Ministral 8B	100	69	33	0	0	40.5%
Mistral Small Creative	100	67	19	0	0	37.1%
Mistral Large 2	64	50	33	0	0	29.5%
Mistral Large	54	52	0	0	0	21.3%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	99	99.8%
GPT-4.1 Nano	100	100	100	100	92	98.3%
Z.AI GLM 5	100	100	100	96	92	97.7%
GPT-4o, Aug. 6th (temp=1)	100	100	100	96	91	97.4%
Gemini 2.5 Flash Lite	100	100	100	100	83	96.7%
Gemini 2.5 Pro	100	100	100	100	83	96.7%
DeepSeek V3.1	100	100	100	100	83	96.7%
Qwen 3.5 397B A17B	100	100	98	94	90	96.5%
MoonshotAI: Kimi K2.5	100	100	99	96	83	95.5%
Gemini 2.5 Flash	100	100	96	95	83	94.7%
GPT-5	100	100	100	83	83	93.3%
Minimax M2.5	100	100	100	83	83	93.3%
Grok 4.1 Fast	100	100	100	83	83	93.3%
Stealth: Aurora Alpha	100	100	100	100	67	93.3%
GPT-5 Nano	100	100	100	83	83	93.3%
ByteDance Seed 1.6	100	97	97	83	83	92.1%
GPT-4o Mini (temp=1)	100	100	98	90	67	90.9%
Claude Opus 4.6	100	100	100	83	67	90.0%
Writer: Palmyra X5	100	100	100	83	67	90.0%
Llama 3.1 8B	100	100	100	100	50	90.0%
DeepSeek V3.2	100	100	97	83	67	89.4%
Qwen 3.5 Plus (2026-02-15)	100	100	97	83	67	89.4%
Claude Sonnet 4	100	98	89	88	70	89.0%
Claude Sonnet 4.5	100	94	89	86	67	87.2%
Grok 4	100	98	86	84	67	87.0%
GPT-5.1	100	100	100	83	50	86.7%
GPT-4.1	100	100	83	83	67	86.7%
Gemma 3 27B	100	100	100	83	50	86.7%
o4 Mini High	100	99	83	83	67	86.4%
Ministral 3 8B	100	100	88	87	50	84.9%
Gemma 3 4B	100	93	82	82	67	84.7%
o4 Mini	88	83	83	83	77	83.1%
DeepSeek V3 (2025-03-24)	100	87	79	79	67	82.4%
GPT-4.1 Mini	90	83	83	81	75	82.4%
Gemma 3 12B	100	93	83	83	50	81.9%
GPT-4o, Aug. 6th (temp=0)	95	83	78	68	67	78.1%
Gemini 3.1 Pro (Preview)	83	81	80	79	67	78.0%
Claude Opus 4.5	100	83	67	67	67	76.7%
Z.AI GLM 4.7 Flash	100	100	67	67	50	76.7%
Hermes 3 70B	100	83	78	62	60	76.6%
Arcee AI: Trinity Mini	100	96	71	67	49	76.5%
GPT-4o, May 13th (temp=1)	100	97	88	50	48	76.4%
Mistral Small Creative	100	94	76	62	50	76.3%
Claude Haiku 4.5	91	83	83	62	50	73.8%
Claude 3.5 Sonnet	84	75	71	71	66	73.4%
Cohere Command R+ (Aug. 2024)	100	75	67	67	58	73.3%
Ministral 8B	100	100	100	50	0	70.0%
Ministral 3 3B	94	83	68	67	33	69.1%
DeepSeek-V2 Chat	100	84	63	50	46	68.6%
Rocinante 12B	100	80	67	53	44	68.5%
Mistral Large 2	96	71	69	61	33	66.1%
Mistral Medium 3.1	100	83	64	63	17	65.6%
Mistral Large 3	98	74	55	50	50	65.5%
GPT-4o Mini (temp=0)	73	67	67	65	52	64.8%
Claude 3.5 Haiku	77	75	68	58	45	64.5%
Claude Opus 4	81	74	70	63	33	64.3%
Gemini 3 Flash (Preview)	83	83	67	50	33	63.3%
Z.AI GLM 4.7	81	67	59	50	50	61.2%
Mistral NeMO	86	83	83	50	0	60.5%
GPT-5.2	83	67	67	50	33	60.0%
WizardLM 2 8x22b	100	100	50	50	0	60.0%
Mistral Large	72	67	67	58	33	59.4%
Z.AI GLM 4.5	99	87	54	33	17	57.9%
Ministral 3 14B	82	56	50	50	50	57.6%
DeepSeek V3 (2024-12-26)	70	67	53	48	46	56.7%
Mistral Small 3.2 24B	99	50	50	50	33	56.5%
Ministral 3B	97	58	58	50	17	56.1%
Gemini 3 Pro (Preview)	67	50	50	50	50	53.3%
Llama 3.1 Nemotron 70B	72	69	64	32	21	51.6%
Claude 3 Haiku	78	58	42	40	26	48.7%
Llama 3.1 70B	83	55	49	43	0	46.2%
Qwen 2.5 72B	94	50	50	33	0	45.4%
GPT-4o, May 13th (temp=0)	65	61	59	17	0	40.3%
Arcee AI: Trinity Large (Preview)	90	50	43	17	0	39.9%
Hermes 3 405B	93	48	30	17	0	37.5%
ByteDance Seed 1.6 Flash	67	33	33	0	0	26.7%
Claude 3.7 Sonnet	36	23	17	17	0	18.3%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	100	100	100	93	89	96.4%
Z.AI GLM 4.6	100	100	100	88	77	93.1%
Gemini 2.5 Pro	100	100	100	97	67	92.7%
Gemini 2.5 Flash	100	100	100	92	67	91.7%
GPT-5	100	100	86	83	78	89.6%
Gemma 3 4B	100	100	83	83	80	89.3%
GPT-5.1	100	100	100	83	49	86.5%
Claude Opus 4.6	100	83	83	83	67	83.4%
GPT-5 Nano	100	100	83	83	50	83.3%
Llama 3.1 8B	100	100	100	67	47	82.8%
Grok 4	91	91	90	71	70	82.6%
Gemini 2.5 Flash Lite	100	83	80	73	68	81.0%
Stealth: Aurora Alpha	100	90	77	70	67	80.7%
GPT-4.1 Nano	86	83	82	81	69	80.2%
GPT-5 Mini	100	100	83	83	33	80.0%
DeepSeek V3.1	95	86	81	67	67	78.9%
Grok 4 Fast	91	81	78	73	66	77.7%
Gemma 3 12B	100	88	71	67	60	77.1%
Arcee AI: Trinity Mini	100	83	77	76	48	77.0%
Gemini 3 Flash (Preview)	96	76	75	67	65	75.6%
Claude 3.5 Haiku	97	91	73	64	44	74.1%
Llama 3.1 Nemotron 70B	82	73	73	70	66	72.6%
Arcee AI: Trinity Large (Preview)	92	91	78	67	33	72.1%
Minimax M2.5	85	77	68	65	62	71.2%
ByteDance Seed 1.6	85	83	67	67	50	70.4%
Claude Opus 4.5	90	83	74	50	50	69.4%
Mistral Small 3.2 24B	100	100	67	58	13	67.5%
o4 Mini High	83	77	69	56	50	67.2%
Qwen 3.5 397B A17B	83	83	83	50	33	66.5%
WizardLM 2 8x22b	87	78	67	50	47	65.6%
DeepSeek V3 (2024-12-26)	81	73	72	57	43	65.6%
Grok 4.1 Fast	84	80	63	51	50	65.6%
GPT-4.1	96	67	61	50	50	64.6%
Mistral NeMO	83	73	70	63	33	64.5%
Ministral 3 8B	76	70	67	58	50	64.2%
DeepSeek-V2 Chat	67	62	59	53	51	58.5%
DeepSeek V3.2	67	67	59	50	50	58.5%
Ministral 3 14B	79	60	59	50	42	58.1%
Gemini 3 Pro (Preview)	86	54	50	50	50	57.9%
Hermes 3 70B	87	67	56	56	23	57.6%
Ministral 3B	73	69	59	47	35	56.7%
GPT-4o, Aug. 6th (temp=1)	95	65	52	46	25	56.4%
ByteDance Seed 1.6 Flash	62	61	60	50	48	56.2%
Z.AI GLM 5	83	81	78	33	0	55.3%
Claude Opus 4	74	67	52	48	33	55.0%
Writer: Palmyra X5	80	67	63	46	17	54.5%
GPT-4.1 Mini	83	79	46	33	31	54.4%
Gemma 3 27B	72	65	52	42	40	54.2%
Claude Haiku 4.5	67	66	53	50	33	54.0%
Claude 3.7 Sonnet	65	61	55	50	39	53.9%
Ministral 8B	73	65	60	52	17	53.1%
Mistral Medium 3.1	71	50	50	47	47	53.1%
Mistral Large	67	67	63	41	27	52.9%
Llama 3.1 70B	74	64	50	39	33	52.1%
Qwen 2.5 72B	61	54	53	50	41	51.8%
Hermes 3 405B	67	66	52	50	23	51.7%
Claude Sonnet 4	62	62	58	55	18	51.1%
MoonshotAI: Kimi K2.5	81	58	49	33	33	50.9%
Z.AI GLM 4.7 Flash	70	67	50	33	33	50.6%
Ministral 3 3B	57	55	50	49	35	49.2%
GPT-4o, Aug. 6th (temp=0)	71	51	43	41	38	48.8%
o4 Mini	67	50	47	44	33	48.2%
GPT-4o Mini (temp=1)	80	50	44	41	25	47.9%
Claude Sonnet 4.5	65	56	52	38	29	47.9%
DeepSeek V3 (2025-03-24)	70	52	47	45	23	47.1%
Z.AI GLM 4.5	77	53	37	33	33	46.6%
Mistral Large 2	58	51	50	39	33	46.1%
Claude 3.5 Sonnet	55	51	47	44	33	45.9%
Claude 3 Haiku	71	55	50	33	16	45.0%
Mistral Large 3	58	49	47	36	26	43.1%
Mistral Small Creative	63	50	33	32	28	41.1%
Rocinante 12B	76	50	43	37	0	41.0%
Gemini 3.1 Pro (Preview)	58	50	43	33	17	40.2%
Cohere Command R+ (Aug. 2024)	49	47	47	42	16	40.0%
Qwen 3.5 Plus (2026-02-15)	83	50	33	17	17	40.0%
Z.AI GLM 4.7	78	50	33	30	0	38.4%
GPT-4o, May 13th (temp=1)	43	38	31	29	28	33.8%
GPT-4o, May 13th (temp=0)	67	27	27	26	17	32.7%
GPT-4o Mini (temp=0)	59	50	17	13	0	27.8%
GPT-5.2	16	0	0	0	0	3.2%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5 Mini	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	99	99.8%
Qwen 3.5 397B A17B	100	100	100	100	96	99.3%
Gemini 3 Pro (Preview)	100	100	100	98	96	98.8%
Z.AI GLM 5	100	100	100	100	93	98.6%
MoonshotAI: Kimi K2.5	100	100	100	100	91	98.3%
Gemini 3.1 Pro (Preview)	100	100	100	100	90	98.0%
Grok 4 Fast	100	100	100	100	89	97.7%
Cohere Command R+ (Aug. 2024)	100	100	100	100	87	97.4%
Hermes 3 405B	100	100	100	95	91	97.3%
GPT-4.1	100	100	98	96	91	96.9%
o4 Mini High	100	100	100	100	83	96.7%
Claude Opus 4.5	100	100	100	100	83	96.7%
o4 Mini	100	100	100	100	83	96.7%
ByteDance Seed 1.6	100	100	100	100	83	96.7%
DeepSeek V3.2	100	100	100	100	83	96.7%
Gemini 2.5 Flash Lite	100	100	100	100	83	96.7%
Claude Sonnet 4	100	100	100	99	83	96.4%
Claude Sonnet 4.5	100	100	97	94	80	94.3%
Minimax M2.5	100	100	100	100	67	93.3%
DeepSeek-V2 Chat	100	100	100	86	81	93.3%
Grok 4.1 Fast	100	100	100	83	81	92.9%
Gemma 3 27B	100	100	100	83	79	92.5%
Claude 3.5 Haiku	100	100	100	88	73	92.3%
GPT-4o Mini (temp=1)	100	100	100	90	67	91.3%
GPT-4o, Aug. 6th (temp=1)	100	100	98	85	69	90.5%
Writer: Palmyra X5	100	100	99	83	67	89.9%
Gemini 3 Flash (Preview)	100	100	99	83	67	89.7%
Claude Haiku 4.5	100	100	92	89	67	89.5%
Llama 3.1 Nemotron 70B	100	100	97	88	50	86.8%
Llama 3.1 8B	100	100	100	83	50	86.7%
Mistral Medium 3.1	100	95	84	80	74	86.4%
Claude 3.5 Sonnet	95	88	84	82	76	85.1%
Ministral 8B	95	85	85	82	78	84.8%
Ministral 3 14B	100	90	81	74	70	83.0%
Claude Opus 4	100	100	99	79	33	82.3%
Stealth: Aurora Alpha	100	92	83	67	65	81.4%
DeepSeek V3 (2024-12-26)	100	94	83	76	51	80.6%
Mistral Small 3.2 24B	100	100	100	99	0	79.7%
Z.AI GLM 4.5	100	83	83	81	50	79.4%
Ministral 3B	100	84	79	79	56	79.4%
GPT-4o, May 13th (temp=0)	100	100	79	67	50	79.1%
Gemma 3 12B	100	83	77	67	67	78.8%
Mistral Large	94	84	83	82	50	78.7%
Mistral Small Creative	100	88	82	72	50	78.3%
Ministral 3 8B	98	86	85	77	42	77.6%
Mistral Large 2	91	83	77	69	66	77.1%
Mistral Large 3	93	87	78	67	55	76.1%
Qwen 2.5 72B	83	80	75	67	67	74.4%
DeepSeek V3 (2025-03-24)	86	83	76	72	54	74.2%
GPT-4o, Aug. 6th (temp=0)	100	78	73	67	50	73.6%
GPT-4o, May 13th (temp=1)	100	83	77	74	33	73.5%
Rocinante 12B	91	89	76	67	43	73.3%
Claude 3.7 Sonnet	83	82	79	63	59	73.2%
ByteDance Seed 1.6 Flash	100	83	82	67	17	69.7%
Ministral 3 3B	82	82	70	57	49	68.1%
GPT-4o Mini (temp=0)	100	88	67	50	33	67.5%
Mistral NeMO	100	83	67	66	17	66.5%
GPT-5.2	83	81	75	59	33	66.3%
Llama 3.1 70B	100	63	59	56	41	63.9%
Hermes 3 70B	67	62	57	56	50	58.3%
Arcee AI: Trinity Large (Preview)	100	47	44	33	22	49.2%
Claude 3 Haiku	65	63	52	45	0	45.1%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Grok 4	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
GPT-4.1 Nano	100	100	100	100	100	100.0%
Gemma 3 4B	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	83	96.7%
Arcee AI: Trinity Mini	100	100	100	100	83	96.6%
Grok 4 Fast	100	100	97	83	83	92.7%
DeepSeek V3 (2024-12-26)	100	100	90	83	82	91.2%
ByteDance Seed 1.6	100	100	100	100	50	90.0%
DeepSeek V3.2	100	100	100	83	67	90.0%
GPT-5 Mini	100	100	100	100	33	86.7%
Stealth: Aurora Alpha	100	100	83	77	62	84.6%
Gemma 3 27B	100	96	83	75	65	83.7%
Mistral Small Creative	100	100	92	91	33	83.1%
Cohere Command R+ (Aug. 2024)	100	100	98	87	26	82.2%
Mistral Large 2	100	89	72	69	67	79.4%
GPT-4o Mini (temp=1)	100	100	68	67	50	77.0%
MoonshotAI: Kimi K2.5	100	100	83	83	17	76.7%
Z.AI GLM 4.7 Flash	100	83	83	80	33	76.0%
Mistral Large 3	100	100	83	56	40	75.8%
GPT-4.1 Mini	83	83	75	67	67	75.1%
o4 Mini	100	100	100	67	0	73.3%
WizardLM 2 8x22b	100	100	79	67	17	72.5%
Llama 3.1 70B	100	94	69	53	46	72.3%
DeepSeek-V2 Chat	100	96	83	71	0	70.0%
GPT-4o Mini (temp=0)	81	78	64	64	61	69.8%
o4 Mini High	100	100	100	48	0	69.5%
GPT-5 Nano	100	83	75	50	33	68.3%
GPT-4o, May 13th (temp=1)	99	90	83	67	0	67.9%
Ministral 3 14B	100	92	79	50	17	67.6%
Grok 4.1 Fast	83	83	67	67	33	66.7%
GPT-4o, May 13th (temp=0)	93	83	78	67	0	64.3%
GPT-4o, Aug. 6th (temp=0)	100	67	51	50	50	63.6%
Gemini 2.5 Flash	100	92	83	33	0	61.6%
Ministral 3B	90	67	50	50	50	61.4%
Claude Sonnet 4	100	100	67	33	0	60.0%
Z.AI GLM 4.5	100	67	67	67	0	60.0%
Mistral Medium 3.1	83	83	81	50	0	59.6%
Gemini 2.5 Flash Lite	100	100	67	25	0	58.4%
Claude Opus 4.6	100	67	50	50	17	56.7%
Gemini 3 Flash (Preview)	83	67	67	50	17	56.7%
Claude Haiku 4.5	100	100	67	17	0	56.7%
Hermes 3 405B	100	50	50	50	33	56.7%
Mistral Large	100	93	67	17	0	55.2%
Ministral 8B	100	98	67	0	0	52.9%
GPT-4o, Aug. 6th (temp=1)	96	67	50	50	0	52.5%
Rocinante 12B	90	67	61	34	7	51.8%
Mistral NeMO	83	58	50	50	17	51.6%
Ministral 3 8B	100	83	67	0	0	50.0%
Claude 3.5 Haiku	76	62	56	44	8	49.2%
Z.AI GLM 4.7	95	83	67	0	0	49.1%
Z.AI GLM 5	100	75	41	29	0	49.1%
Claude 3 Haiku	70	63	56	33	19	48.2%
Hermes 3 70B	100	83	50	0	0	46.7%
Qwen 3.5 397B A17B	100	83	33	17	0	46.7%
GPT-5	100	67	50	17	0	46.7%
Llama 3.1 8B	100	67	33	32	0	46.4%
Gemma 3 12B	98	67	67	0	0	46.2%
GPT-4.1	83	61	33	33	0	42.3%
Gemini 3 Pro (Preview)	83	67	33	17	0	40.0%
Writer: Palmyra X5	83	50	50	17	0	40.0%
Arcee AI: Trinity Large (Preview)	90	75	26	0	0	38.1%
Qwen 3.5 Plus (2026-02-15)	67	50	50	17	0	36.7%
Llama 3.1 Nemotron 70B	64	33	33	33	17	36.0%
DeepSeek V3 (2025-03-24)	100	50	11	0	0	32.2%
ByteDance Seed 1.6 Flash	100	33	17	10	0	32.0%
Claude 3.7 Sonnet	56	50	33	17	0	31.2%
Claude 3.5 Sonnet	75	67	3	0	0	28.9%
Gemini 3.1 Pro (Preview)	45	33	33	17	0	25.6%
Mistral Small 3.2 24B	82	17	17	0	0	23.1%
Ministral 3 3B	43	33	33	0	0	22.0%
Claude Opus 4	83	17	0	0	0	20.0%
Claude Opus 4.5	50	17	17	17	0	20.0%
Qwen 2.5 72B	91	0	0	0	0	18.2%
Minimax M2.5	33	17	17	17	6	17.9%
Claude Sonnet 4.5	83	0	0	0	0	16.7%
Claude Sonnet 4.6	48	17	17	0	0	16.3%
GPT-5.1	0	0	0	0	0	0.0%
GPT-5.2	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Sonnet 4.6	100	100	100	100	83	96.6%
Gemini 2.5 Pro	100	96	90	83	83	90.6%
ByteDance Seed 1.6	100	100	83	83	83	90.0%
DeepSeek V3.1	100	97	90	83	67	87.4%
Z.AI GLM 4.6	100	100	87	79	59	84.9%
GPT-5 Nano	100	83	83	83	67	83.3%
GPT-4.1 Nano	100	90	76	67	50	76.5%
DeepSeek V3.2	91	78	78	78	51	75.2%
Cohere Command R+ (Aug. 2024)	99	98	87	51	33	73.6%
GPT-5	100	100	67	50	50	73.3%
Claude Opus 4.6	100	98	67	50	50	73.0%
Gemma 3 4B	100	100	67	62	33	72.4%
Mistral Medium 3.1	90	88	67	60	27	66.1%
Grok 4 Fast	96	71	66	50	33	63.3%
GPT-4.1	83	83	50	50	48	62.8%
Mistral Small Creative	95	66	62	50	33	61.3%
Gemini 2.5 Flash Lite	86	69	67	46	33	60.1%
GPT-4o, Aug. 6th (temp=1)	90	67	62	50	33	60.1%
Stealth: Aurora Alpha	67	64	58	50	50	57.7%
Mistral Small 3.2 24B	83	73	67	33	17	54.7%
Claude Sonnet 4.5	75	67	66	42	17	53.4%
GPT-5.1	100	67	50	33	17	53.3%
Z.AI GLM 5	100	83	50	17	17	53.3%
Mistral Large	65	62	55	45	33	52.1%
Gemini 2.5 Flash	67	64	51	41	33	51.3%
Rocinante 12B	83	72	62	38	1	51.3%
o4 Mini High	77	75	67	33	0	50.4%
Qwen 3.5 Plus (2026-02-15)	84	57	51	41	17	49.8%
Mistral NeMO	98	54	50	28	17	49.3%
Ministral 3 8B	74	62	62	46	0	48.8%
Claude Sonnet 4	63	58	49	37	31	47.6%
WizardLM 2 8x22b	67	67	54	50	0	47.4%
Claude Opus 4.5	69	65	50	45	3	46.3%
Minimax M2.5	83	67	50	17	0	43.3%
Claude Haiku 4.5	67	67	38	26	17	43.0%
GPT-4o Mini (temp=1)	56	53	36	33	33	42.4%
Grok 4	67	67	50	26	0	41.9%
Claude 3.5 Haiku	77	69	30	18	12	41.2%
Ministral 8B	68	62	48	5	0	36.6%
Claude Opus 4	52	43	42	33	10	36.2%
Arcee AI: Trinity Large (Preview)	74	60	31	0	0	33.1%
Gemma 3 12B	83	39	25	9	6	32.5%
Ministral 3B	68	54	32	8	0	32.4%
Gemini 3 Flash (Preview)	50	49	33	17	0	29.9%
DeepSeek-V2 Chat	67	50	17	10	0	28.6%
Hermes 3 70B	73	47	16	7	0	28.5%
DeepSeek V3 (2024-12-26)	60	42	30	8	0	27.9%
Claude 3.5 Sonnet	44	42	27	17	7	27.6%
MoonshotAI: Kimi K2.5	56	33	33	11	0	26.9%
GPT-5 Mini	67	33	17	17	0	26.7%
Mistral Large 2	53	32	32	16	0	26.7%
Writer: Palmyra X5	83	47	0	0	0	26.0%
Mistral Large 3	57	44	17	9	0	25.4%
GPT-4o, Aug. 6th (temp=0)	65	59	0	0	0	24.9%
o4 Mini	74	33	17	0	0	24.7%
Arcee AI: Trinity Mini	46	39	26	10	0	23.9%
Qwen 3.5 397B A17B	35	33	33	17	0	23.7%
Z.AI GLM 4.7 Flash	42	32	24	17	0	23.0%
ByteDance Seed 1.6 Flash	81	17	14	0	0	22.4%
Llama 3.1 70B	67	33	0	0	0	20.0%
Ministral 3 14B	49	47	2	0	0	19.8%
Gemma 3 27B	67	16	0	0	0	16.5%
Llama 3.1 8B	50	29	0	0	0	15.7%
Gemini 3.1 Pro (Preview)	31	17	17	10	0	15.0%
Z.AI GLM 4.7	50	24	0	0	0	14.8%
Grok 4.1 Fast	47	17	8	0	0	14.3%
Ministral 3 3B	22	20	16	11	0	13.8%
Gemini 3 Pro (Preview)	43	23	0	0	0	13.2%
GPT-5.2	17	17	17	15	0	13.0%
Hermes 3 405B	29	20	15	0	0	12.8%
GPT-4.1 Mini	57	0	0	0	0	11.4%
Qwen 2.5 72B	25	17	0	0	0	8.4%
DeepSeek V3 (2025-03-24)	26	14	0	0	0	7.9%
GPT-4o, May 13th (temp=1)	17	12	0	0	0	5.8%
Claude 3.7 Sonnet	28	0	0	0	0	5.5%
Llama 3.1 Nemotron 70B	23	1	0	0	0	4.9%
GPT-4o, May 13th (temp=0)	17	0	0	0	0	3.3%
Claude 3 Haiku	13	0	0	0	0	2.6%
Z.AI GLM 4.5	10	0	0	0	0	1.9%
GPT-4o Mini (temp=0)	0	0	0	0	0	0.0%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	90	98.1%
GPT-5.1	100	100	100	100	83	96.7%
Minimax M2.5	100	100	100	100	83	96.7%
Grok 4	100	100	100	100	83	96.7%
Claude Sonnet 4.6	100	100	100	100	83	96.7%
ByteDance Seed 1.6	100	100	100	100	83	96.7%
GPT-4.1 Nano	100	100	100	100	83	96.7%
Z.AI GLM 5	100	100	100	100	83	96.7%
Claude Opus 4.6	100	100	100	98	83	96.3%
Gemma 3 27B	100	100	100	97	83	96.1%
MoonshotAI: Kimi K2.5	100	100	100	94	81	95.1%
Mistral Small Creative	100	100	97	93	84	94.8%
GPT-5 Mini	100	100	100	83	83	93.3%
o4 Mini	100	100	100	83	83	93.3%
Claude Opus 4	100	100	100	83	83	93.3%
DeepSeek V3.2	100	100	100	100	67	93.3%
Gemini 2.5 Flash	100	100	100	83	83	93.3%
GPT-4o Mini (temp=0)	100	100	100	83	83	93.3%
Claude Opus 4.5	100	100	95	83	83	92.3%
o4 Mini High	100	100	100	100	61	92.3%
GPT-4.1	100	96	94	86	84	92.0%
DeepSeek V3 (2025-03-24)	100	100	87	83	83	90.8%
Gemma 3 4B	100	99	95	93	67	90.7%
GPT-4o, May 13th (temp=0)	100	100	100	100	50	90.0%
Gemini 2.5 Flash Lite	100	100	83	83	83	90.0%
WizardLM 2 8x22b	100	100	100	83	67	90.0%
Z.AI GLM 4.7	100	100	96	83	70	89.8%
Gemini 3 Pro (Preview)	100	95	83	83	78	87.9%
GPT-4o, Aug. 6th (temp=1)	100	100	100	83	55	87.6%
GPT-5	100	100	100	83	50	86.7%
Z.AI GLM 4.6	100	100	100	100	33	86.6%
GPT-4o, Aug. 6th (temp=0)	100	100	79	78	66	84.6%
GPT-4.1 Mini	100	98	95	79	50	84.3%
Gemini 3 Flash (Preview)	95	94	86	83	59	83.4%
DeepSeek-V2 Chat	100	100	78	67	56	80.0%
ByteDance Seed 1.6 Flash	100	94	89	83	33	79.9%
Claude 3.5 Haiku	86	85	85	80	57	78.7%
Claude Sonnet 4	100	90	89	83	21	76.7%
Qwen 3.5 Plus (2026-02-15)	96	79	78	73	50	75.4%
Claude Haiku 4.5	100	100	87	70	17	74.8%
Grok 4.1 Fast	100	100	88	83	0	74.2%
Writer: Palmyra X5	100	100	95	67	0	72.3%
Claude 3.5 Sonnet	87	77	73	59	54	70.0%
Llama 3.1 8B	100	97	67	58	28	69.8%
Stealth: Aurora Alpha	83	79	75	61	50	69.6%
Llama 3.1 70B	87	83	67	61	50	69.5%
GPT-4o, May 13th (temp=1)	97	92	65	56	33	68.7%
Cohere Command R+ (Aug. 2024)	90	83	83	55	24	67.1%
Mistral NeMO	94	84	79	76	0	66.7%
Ministral 3 8B	93	91	58	48	33	64.7%
Ministral 8B	77	75	67	66	33	63.8%
Mistral Large	100	77	69	53	20	63.8%
GPT-5 Nano	100	67	50	50	50	63.3%
DeepSeek V3 (2024-12-26)	100	100	67	46	2	63.0%
Ministral 3 14B	100	73	53	50	31	61.3%
Claude Sonnet 4.5	100	92	74	20	0	57.2%
Gemini 3.1 Pro (Preview)	76	69	67	43	30	57.0%
Llama 3.1 Nemotron 70B	67	63	55	52	44	56.0%
Claude 3.7 Sonnet	82	72	50	37	33	54.8%
Gemma 3 12B	100	67	49	33	17	53.2%
Rocinante 12B	100	88	67	0	0	50.8%
Arcee AI: Trinity Large (Preview)	80	73	50	37	7	49.4%
Qwen 2.5 72B	76	55	47	32	0	42.2%
Hermes 3 405B	82	72	56	0	0	42.1%
Z.AI GLM 4.5	65	60	50	29	0	40.8%
Mistral Large 3	87	66	47	0	0	40.2%
Hermes 3 70B	83	65	21	7	0	35.1%
Mistral Large 2	83	59	0	0	0	28.5%
Ministral 3 3B	100	17	13	0	0	25.9%
Mistral Small 3.2 24B	49	33	23	23	0	25.6%
Qwen 3.5 397B A17B	50	41	33	0	0	24.9%
Mistral Medium 3.1	100	17	5	0	0	24.4%
Claude 3 Haiku	53	33	32	0	0	23.5%
GPT-5.2	43	29	15	0	0	17.5%
Ministral 3B	38	3	0	0	0	8.1%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 2.5 Pro	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	99.9%
GPT-5	100	100	100	100	96	99.2%
MoonshotAI: Kimi K2.5	100	100	100	100	93	98.7%
Gemini 2.5 Flash	100	100	100	100	83	96.7%
Z.AI GLM 4.7 Flash	100	100	100	96	83	95.8%
Z.AI GLM 5	100	100	100	99	78	95.4%
Z.AI GLM 4.6	100	100	100	90	86	95.3%
GPT-4.1 Nano	100	100	100	84	83	93.5%
Gemma 3 27B	100	100	100	83	67	90.0%
GPT-5 Mini	100	100	100	82	67	89.7%
Gemini 2.5 Flash Lite	100	100	100	83	61	88.8%
DeepSeek V3 (2024-12-26)	100	100	95	83	60	87.6%
o4 Mini High	100	100	83	83	67	86.7%
GPT-5 Nano	100	100	83	83	67	86.7%
Claude Opus 4.6	100	100	83	83	67	86.7%
GPT-5.1	100	100	97	83	50	86.2%
Writer: Palmyra X5	100	94	83	83	67	85.4%
GPT-4.1 Mini	100	93	83	81	67	84.9%
ByteDance Seed 1.6	100	100	88	83	50	84.4%
Claude Opus 4.5	100	91	83	80	67	84.3%
DeepSeek-V2 Chat	100	83	80	78	77	83.6%
Rocinante 12B	100	100	90	76	50	83.3%
Claude Haiku 4.5	95	93	83	80	63	82.5%
Mistral Small Creative	93	91	83	76	67	82.1%
Grok 4 Fast	99	94	83	67	67	82.1%
GPT-4o, Aug. 6th (temp=1)	100	100	93	83	33	82.0%
Claude Sonnet 4.5	100	84	80	79	67	81.9%
Gemini 3.1 Pro (Preview)	100	100	81	67	59	81.4%
Grok 4.1 Fast	99	91	84	67	59	79.8%
WizardLM 2 8x22b	100	98	83	67	50	79.5%
Minimax M2.5	100	100	74	71	50	78.9%
Mistral Medium 3.1	95	83	79	67	67	78.1%
Mistral Small 3.2 24B	100	100	99	83	0	76.5%
GPT-4o Mini (temp=1)	99	94	73	68	49	76.4%
Hermes 3 70B	100	100	83	50	48	76.2%
DeepSeek V3 (2025-03-24)	88	88	78	72	46	74.3%
GPT-4.1	100	82	72	67	50	74.2%
Z.AI GLM 4.7	100	95	67	59	33	70.9%
Ministral 3 8B	83	77	74	61	50	68.8%
ByteDance Seed 1.6 Flash	100	94	82	67	0	68.6%
Claude Sonnet 4	88	83	76	61	33	68.1%
Grok 4	82	67	67	63	61	68.0%
Claude 3.5 Haiku	81	76	70	56	55	67.5%
GPT-4o, Aug. 6th (temp=0)	89	81	58	53	50	66.4%
Mistral Large 3	80	74	64	64	48	66.2%
o4 Mini	83	67	67	62	50	65.8%
Gemini 3 Pro (Preview)	89	83	77	67	0	63.1%
Arcee AI: Trinity Large (Preview)	78	74	60	51	51	62.6%
Gemma 3 4B	83	67	67	50	45	62.6%
GPT-4o, May 13th (temp=1)	100	65	50	49	46	62.1%
Claude Opus 4	84	75	61	48	42	62.0%
Gemini 3 Flash (Preview)	83	76	67	50	33	61.9%
Stealth: Aurora Alpha	83	66	58	50	50	61.5%
Cohere Command R+ (Aug. 2024)	100	67	64	33	28	58.4%
Gemma 3 12B	83	75	67	67	0	58.3%
Arcee AI: Trinity Mini	87	83	65	50	0	56.9%
Ministral 8B	83	79	56	50	16	56.8%
Mistral NeMO	67	64	54	49	33	53.4%
Mistral Large 2	90	50	48	48	23	51.9%
GPT-4o Mini (temp=0)	72	59	47	43	34	50.9%
Qwen 3.5 397B A17B	69	67	50	50	17	50.6%
Qwen 3.5 Plus (2026-02-15)	83	61	50	33	17	48.8%
Claude 3.5 Sonnet	64	57	54	41	28	48.8%
Z.AI GLM 4.5	69	67	40	33	33	48.5%
Ministral 3 14B	78	67	46	33	17	48.3%
Ministral 3 3B	71	50	50	50	17	47.5%
Llama 3.1 8B	100	67	50	19	0	47.1%
Ministral 3B	87	49	34	30	27	45.5%
Qwen 2.5 72B	53	50	49	38	33	44.8%
Mistral Large	63	51	50	29	24	43.5%
Llama 3.1 70B	57	55	44	31	24	41.9%
Llama 3.1 Nemotron 70B	56	46	43	30	28	40.3%
Hermes 3 405B	55	48	48	44	0	39.0%
Claude 3.7 Sonnet	50	50	33	17	16	33.3%
GPT-4o, May 13th (temp=0)	50	43	39	29	0	32.1%
GPT-5.2	50	33	32	17	0	26.3%
Claude 3 Haiku	37	4	2	0	0	8.7%

Name drop frequency

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets