Narrator intent-glossing

Test: Bad Writing Habits

Avg. Score

69.4%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Grok 4.1 Fast	97.8%	$0.0018	37.8s	81%
2	o4 Mini	92.2%	$0.015	25.7s	68%
3	o4 Mini High	92.7%	$0.025	47.2s	61%
4	DeepSeek V3 (2025-03-24)	85.8%	$0.0014	39.4s	50%
5	ByteDance Seed 1.6 Flash	84.2%	$0.0013	27.3s	48%
6	Hermes 3 405B	89.5%	$0.0032	53.2s	50%
7	Mistral Small Creative	81.2%	$0.0007	9.1s	45%
8	Claude 3 Haiku	83.6%	$0.0025	14.9s	41%
9	Rocinante 12B	85.7%	$0.0014	38.4s	42%
10	Mistral NeMO	82.1%	$0.0005	10.1s	36%
11	Writer: Palmyra X5	84.4%	$0.011	22.0s	41%
12	Mistral Medium 3.1	83.0%	$0.0048	36.5s	42%
13	Mistral Large	80.7%	$0.014	30.9s	42%
14	Grok 4 Fast	77.9%	$0.0017	24.1s	37%
15	Ministral 3 14B	75.1%	$0.0007	11.7s	33%
16	Qwen 2.5 72B	77.4%	$0.0010	36.7s	34%
17	GPT-4.1	79.6%	$0.018	44.7s	40%
18	GPT-4o, May 13th (temp=0)	81.6%	$0.035	14.1s	36%
19	Gemini 3.1 Pro (Preview)	95.7%	$0.107	1.8m	72%
20	DeepSeek V3 (2024-12-26)	78.1%	$0.0021	54.6s	35%
21	DeepSeek-V2 Chat	78.1%	$0.0021	53.3s	34%
22	Hermes 3 70B	80.5%	$0.0010	1.2m	36%
23	Mistral Large 3	74.6%	$0.0033	30.3s	31%
24	Qwen 3.5 397B A17B	89.1%	$0.014	3.0m	58%
25	GPT-4o Mini (temp=1)	74.3%	$0.0012	34.8s	31%
26	GPT-4o Mini (temp=0)	73.6%	$0.0012	34.8s	30%
27	Ministral 3 3B	70.8%	$0.0005	11.1s	24%
28	Ministral 3B	70.4%	$0.0001	8.1s	22%
29	Mistral Large 2	72.3%	$0.013	29.4s	28%
30	Ministral 3 8B	70.1%	$0.0008	19.6s	21%
31	Gemma 3 12B	68.9%	$0.0004	41.3s	24%
32	GPT-4o, Aug. 6th (temp=0)	72.1%	$0.023	22.7s	24%
33	GPT-4o, May 13th (temp=1)	71.4%	$0.033	14.4s	27%
34	DeepSeek V3.2	75.6%	$0.0014	1.9m	34%
35	Qwen 3.5 Plus (2026-02-15)	67.8%	$0.0060	31.5s	23%
36	Ministral 8B	65.3%	$0.0004	10.4s	18%
37	Arcee AI: Trinity Mini	66.1%	$0.0003	9.2s	16%
38	Gemma 3 27B	68.7%	$0.0006	52.6s	23%
39	Gemini 2.5 Pro	72.1%	$0.036	36.2s	29%
40	Z.AI GLM 4.7	72.6%	$0.010	1.4m	31%
41	Cohere Command R+ (Aug. 2024)	74.3%	$0.020	52.5s	24%
42	Gemma 3 4B	63.6%	$0.0002	20.0s	16%
43	Gemini 3 Flash (Preview)	61.2%	$0.0078	19.6s	21%
44	GPT-4.1 Mini	63.4%	$0.0027	19.0s	15%
45	MoonshotAI: Kimi K2.5	82.7%	$0.019	3.2m	45%
46	Gemini 2.5 Flash	58.6%	$0.0052	10.6s	18%
47	GPT-5.1	79.4%	$0.054	1.8m	41%
48	Gemini 3 Pro (Preview)	74.3%	$0.055	54.4s	32%
49	Claude Sonnet 4.6	69.8%	$0.031	39.3s	23%
50	Z.AI GLM 4.7 Flash	66.1%	$0.0017	1.2m	23%
51	GPT-4o, Aug. 6th (temp=1)	64.3%	$0.018	24.4s	17%
52	ByteDance Seed 1.6	76.7%	$0.013	2.5m	33%
53	Z.AI GLM 5	65.9%	$0.0084	1.2m	22%
54	Z.AI GLM 4.5	61.9%	$0.0051	42.1s	14%
55	Llama 3.1 70B	59.0%	$0.0015	29.4s	12%
56	Claude Sonnet 4.5	64.8%	$0.035	38.1s	21%
57	GPT-4.1 Nano	53.9%	$0.0007	13.3s	11%
58	Arcee AI: Trinity Large (Preview)	58.1%	$0.0000	43.6s	12%
59	Z.AI GLM 4.6	58.5%	$0.0065	51.5s	14%
60	WizardLM 2 8x22b	66.3%	$0.0026	1.8m	19%
61	Gemini 2.5 Flash Lite	48.0%	$0.0009	9.5s	9%
62	DeepSeek V3.1	63.9%	$0.0020	1.8m	19%
63	Grok 4	71.9%	$0.048	1.7m	26%
64	Claude Opus 4.6	68.8%	$0.078	1.2m	29%
65	Claude 3.7 Sonnet	61.0%	$0.042	46.7s	16%
66	Llama 3.1 Nemotron 70B	48.3%	$0.0038	31.7s	9%
67	Minimax M2.5	56.4%	$0.0034	1.3m	11%
68	Claude 3.5 Haiku	48.7%	$0.0035	10.8s	1%
69	Claude 3.5 Sonnet	60.4%	$0.048	35.5s	13%
70	Claude Haiku 4.5	45.4%	$0.011	21.6s	7%
71	Llama 3.1 8B	50.4%	$0.0003	1.3m	9%
72	Claude Sonnet 4	54.4%	$0.032	43.7s	9%
73	GPT-5	72.1%	$0.065	2.8m	31%
74	GPT-5 Mini	39.1%	$0.0100	57.4s	8%
75	Stealth: Aurora Alpha	29.3%	$0.0000	9.8s	0%
76	GPT-5.2	54.5%	$0.056	1.5m	15%
77	Claude Opus 4.5	52.8%	$0.070	53.4s	11%
78	Mistral Small 3.2 24B	80.0%	$0.0069	5.7m	27%
79	Claude Opus 4	71.2%	$0.209	1.4m	29%
80	GPT-5 Nano	10.4%	$0.0042	1.4m	0%
69.41%

Individual Scenarios

Detailed Writing Rules

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
o4 Mini High	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	90	98.0%
Hermes 3 405B	100	100	100	94	93	97.5%
DeepSeek V3.2	100	100	100	100	75	95.1%
DeepSeek V3 (2025-03-24)	100	100	100	100	72	94.4%
Claude 3 Haiku	100	100	100	97	73	94.0%
GPT-5.1	100	100	100	100	56	91.1%
Claude Opus 4.6	100	100	85	82	79	89.0%
Grok 4 Fast	100	100	92	84	60	87.1%
Gemini 3.1 Pro (Preview)	100	100	100	93	36	85.8%
MoonshotAI: Kimi K2.5	100	100	98	64	62	84.8%
o4 Mini	100	100	100	78	44	84.3%
Qwen 3.5 397B A17B	100	93	90	79	59	84.3%
GPT-4.1 Mini	100	100	100	71	38	81.7%
Mistral Large	100	100	100	75	28	80.6%
Ministral 3 14B	100	100	93	84	25	80.5%
GPT-4o Mini (temp=1)	100	100	100	74	27	80.3%
DeepSeek V3 (2024-12-26)	100	100	100	100	0	80.0%
Hermes 3 70B	100	100	93	67	36	79.2%
GPT-4.1	100	90	89	89	25	78.7%
Mistral Small Creative	100	100	89	57	29	75.1%
GPT-4o, Aug. 6th (temp=0)	100	100	100	54	18	74.5%
Mistral NeMO	100	100	98	68	0	73.2%
Gemini 3 Pro (Preview)	100	83	73	68	25	69.9%
ByteDance Seed 1.6	100	89	64	41	36	66.1%
Arcee AI: Trinity Mini	100	100	76	54	0	66.1%
Mistral Large 2	100	96	72	50	8	65.2%
Qwen 2.5 72B	100	71	67	44	38	64.0%
ByteDance Seed 1.6 Flash	100	100	51	47	22	64.0%
Claude Sonnet 4.5	100	100	90	23	0	62.5%
Writer: Palmyra X5	100	100	100	8	0	61.7%
Ministral 3B	100	100	100	0	0	60.0%
GPT-4o, May 13th (temp=1)	100	79	43	41	36	59.8%
Z.AI GLM 4.6	100	85	64	47	0	59.2%
Claude 3.5 Sonnet	100	93	84	18	0	59.2%
GPT-5	85	84	80	42	0	58.3%
Claude Opus 4.5	100	74	61	53	1	57.7%
Cohere Command R+ (Aug. 2024)	100	93	86	0	0	55.8%
Gemini 3 Flash (Preview)	90	76	54	46	0	53.2%
Gemma 3 27B	76	75	69	36	3	52.1%
Grok 4	100	100	59	0	0	51.8%
Mistral Medium 3.1	100	68	55	27	0	49.8%
Gemini 2.5 Pro	100	70	41	33	0	48.9%
Gemini 2.5 Flash	69	69	40	36	27	48.5%
Qwen 3.5 Plus (2026-02-15)	100	93	42	6	0	48.3%
Z.AI GLM 4.7	100	100	36	0	0	47.3%
DeepSeek-V2 Chat	100	100	31	0	0	46.2%
Mistral Large 3	100	77	48	0	0	45.0%
GPT-4o, Aug. 6th (temp=1)	100	71	48	3	0	44.3%
DeepSeek V3.1	90	88	43	0	0	44.0%
GPT-4o, May 13th (temp=0)	100	100	10	0	0	42.1%
Z.AI GLM 5	100	52	50	3	0	41.0%
Ministral 8B	100	100	0	0	0	40.0%
Gemma 3 12B	100	50	44	0	0	38.9%
Llama 3.1 70B	100	93	0	0	0	38.6%
Gemma 3 4B	79	71	38	0	0	37.5%
Llama 3.1 8B	100	83	0	0	0	36.7%
Claude Opus 4	79	60	17	12	0	33.6%
GPT-5 Mini	67	58	20	8	0	30.6%
Claude Sonnet 4.6	59	52	39	0	0	30.0%
Llama 3.1 Nemotron 70B	100	22	15	0	0	27.3%
Z.AI GLM 4.5	100	35	0	0	0	26.9%
GPT-4o Mini (temp=0)	68	57	5	0	0	26.1%
Arcee AI: Trinity Large (Preview)	100	23	0	0	0	24.7%
GPT-4.1 Nano	100	14	0	0	0	22.7%
Rocinante 12B	100	7	0	0	0	21.4%
Ministral 3 8B	98	0	0	0	0	19.6%
Ministral 3 3B	90	0	0	0	0	18.1%
Gemini 2.5 Flash Lite	50	35	0	0	0	16.9%
Claude 3.7 Sonnet	76	0	0	0	0	15.3%
WizardLM 2 8x22b	55	10	0	0	0	12.9%
Z.AI GLM 4.7 Flash	40	14	0	0	0	10.8%
Minimax M2.5	48	6	0	0	0	10.7%
Stealth: Aurora Alpha	39	0	0	0	0	7.8%
GPT-5.2	17	3	0	0	0	4.1%
Claude Haiku 4.5	7	3	1	0	0	2.3%
Claude Sonnet 4	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
Mistral Small 3.2 24B	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	94	93	97.4%
Z.AI GLM 4.7	100	100	100	100	76	95.1%
GPT-4o, May 13th (temp=0)	100	100	100	88	88	95.1%
Claude 3.5 Sonnet	100	100	100	100	62	92.5%
DeepSeek V3 (2025-03-24)	100	100	100	100	61	92.1%
Rocinante 12B	100	100	100	100	57	91.5%
GPT-4o, Aug. 6th (temp=0)	100	100	100	79	78	91.2%
Qwen 3.5 397B A17B	100	100	98	89	63	90.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	80	65	89.0%
WizardLM 2 8x22b	100	100	100	96	45	88.2%
ByteDance Seed 1.6 Flash	100	100	90	88	61	87.7%
DeepSeek-V2 Chat	100	100	100	81	54	86.9%
o4 Mini	100	100	100	81	46	85.3%
GPT-4.1	100	88	88	83	65	84.8%
Mistral Large 3	100	100	89	79	54	84.3%
Ministral 3 8B	100	100	100	65	54	83.8%
GPT-4.1 Mini	100	100	82	78	59	83.8%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	16	83.2%
Writer: Palmyra X5	100	100	100	65	48	82.6%
Mistral Small Creative	100	100	87	67	56	81.8%
Z.AI GLM 5	100	100	100	71	35	81.1%
DeepSeek V3.2	100	100	98	82	24	80.7%
MoonshotAI: Kimi K2.5	100	100	100	73	29	80.4%
Grok 4 Fast	100	100	94	82	27	80.4%
Mistral Large	100	100	100	100	0	80.0%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
GPT-5	100	100	84	65	45	78.7%
DeepSeek V3 (2024-12-26)	100	100	100	61	30	78.0%
Claude Opus 4	100	100	100	50	40	77.9%
Gemini 3 Pro (Preview)	100	100	81	76	27	76.7%
GPT-4o, May 13th (temp=1)	100	100	99	46	33	75.5%
Mistral Large 2	100	100	71	56	48	74.9%
Claude Opus 4.5	100	80	72	72	50	74.7%
Claude 3 Haiku	100	100	100	57	11	73.7%
Gemini 2.5 Pro	100	75	74	61	57	73.5%
Arcee AI: Trinity Large (Preview)	100	100	96	55	17	73.5%
GPT-5.2	100	95	69	52	48	72.9%
Claude Sonnet 4.5	100	100	88	76	0	72.8%
Claude Sonnet 4.6	100	100	100	57	3	72.1%
Claude Opus 4.6	97	85	81	76	20	71.9%
Ministral 3 14B	100	96	96	44	16	70.2%
Qwen 2.5 72B	95	89	82	75	0	68.1%
GPT-4o Mini (temp=0)	100	86	85	53	15	67.7%
Claude Sonnet 4	100	92	67	65	14	67.5%
DeepSeek V3.1	100	100	100	27	6	66.5%
ByteDance Seed 1.6	100	73	57	54	48	66.5%
Hermes 3 405B	100	100	84	48	0	66.4%
Z.AI GLM 4.7 Flash	96	93	90	43	0	64.3%
Mistral NeMO	100	100	88	33	0	64.2%
Hermes 3 70B	100	100	48	48	22	63.5%
Z.AI GLM 4.6	100	100	100	0	0	60.0%
Z.AI GLM 4.5	100	100	100	0	0	60.0%
Claude 3.5 Haiku	100	100	100	0	0	60.0%
Gemma 3 4B	88	84	59	39	22	58.3%
Gemini 2.5 Flash	100	85	51	48	0	56.9%
Gemma 3 27B	86	72	60	50	15	56.5%
Gemma 3 12B	100	100	56	25	0	56.3%
Arcee AI: Trinity Mini	100	100	81	0	0	56.1%
Ministral 8B	100	92	71	6	0	53.7%
Claude Haiku 4.5	100	90	73	0	0	52.8%
Gemini 3 Flash (Preview)	100	100	64	0	0	52.8%
Grok 4	100	81	50	21	11	52.6%
Cohere Command R+ (Aug. 2024)	100	100	36	25	0	52.3%
Ministral 3 3B	100	81	46	25	0	50.3%
Ministral 3B	100	72	57	7	0	47.3%
Claude 3.7 Sonnet	100	73	34	16	3	45.1%
Llama 3.1 70B	100	84	29	3	0	43.2%
GPT-4o Mini (temp=1)	100	50	33	28	0	42.2%
Minimax M2.5	100	73	15	0	0	37.6%
GPT-5 Mini	89	34	23	0	0	29.1%
GPT-4.1 Nano	82	25	11	0	0	23.7%
Llama 3.1 Nemotron 70B	79	18	6	0	0	20.6%
Gemini 2.5 Flash Lite	100	1	0	0	0	20.2%
GPT-5 Nano	12	9	5	0	0	5.2%
Llama 3.1 8B	17	0	0	0	0	3.3%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	99	99.8%
Claude 3 Haiku	100	100	100	100	99	99.8%
Z.AI GLM 4.7	100	100	100	100	87	97.3%
Claude Sonnet 4.6	100	100	99	94	92	97.1%
DeepSeek V3 (2024-12-26)	100	100	100	100	82	96.5%
Writer: Palmyra X5	100	100	100	96	86	96.3%
o4 Mini	100	100	100	100	79	95.7%
Z.AI GLM 4.6	100	100	100	100	78	95.5%
Hermes 3 70B	100	100	100	96	81	95.2%
Grok 4	100	100	100	100	72	94.4%
Ministral 3 3B	100	100	100	100	72	94.4%
Claude Opus 4	100	100	100	90	78	93.6%
Minimax M2.5	100	100	100	93	72	93.0%
Claude Sonnet 4.5	100	100	100	82	80	92.2%
Arcee AI: Trinity Mini	100	100	100	100	59	91.8%
Gemini 3 Pro (Preview)	100	100	100	100	59	91.7%
Grok 4 Fast	100	100	100	100	56	91.3%
GPT-5	100	100	96	88	72	91.1%
GPT-4.1	100	100	100	100	54	90.8%
Mistral NeMO	100	100	100	85	68	90.5%
WizardLM 2 8x22b	100	100	100	100	50	90.0%
Claude Sonnet 4	100	100	100	82	65	89.4%
Rocinante 12B	100	100	100	100	46	89.3%
Claude 3.7 Sonnet	100	100	100	100	46	89.2%
MoonshotAI: Kimi K2.5	100	100	100	84	56	88.0%
GPT-4o Mini (temp=0)	100	100	100	100	40	87.9%
Z.AI GLM 5	100	100	92	74	72	87.6%
Mistral Large	100	100	100	96	38	86.7%
Qwen 2.5 72B	100	100	92	74	61	85.4%
DeepSeek V3.1	100	91	82	79	62	82.6%
GPT-4o, May 13th (temp=1)	100	100	100	69	36	81.1%
Ministral 3 8B	100	100	100	59	44	80.4%
Claude 3.5 Sonnet	100	100	100	100	0	80.0%
Hermes 3 405B	100	100	100	100	0	80.0%
Ministral 8B	100	100	100	100	0	80.0%
DeepSeek-V2 Chat	100	100	72	65	62	79.9%
Arcee AI: Trinity Large (Preview)	100	100	100	99	0	79.8%
Claude Opus 4.5	100	100	96	82	20	79.6%
Claude Opus 4.6	100	100	100	57	40	79.4%
Gemini 2.5 Pro	100	100	76	67	50	78.5%
Mistral Large 3	100	100	100	77	13	78.0%
Stealth: Aurora Alpha	100	95	94	75	23	77.5%
Ministral 3B	100	100	93	63	31	77.4%
Qwen 3.5 Plus (2026-02-15)	100	100	100	85	0	77.0%
DeepSeek V3.2	100	94	85	81	23	76.6%
o4 Mini High	100	100	100	80	0	75.9%
Gemini 2.5 Flash	100	100	82	69	18	73.9%
GPT-4o, May 13th (temp=0)	100	100	100	52	1	70.5%
Ministral 3 14B	100	100	99	34	17	69.9%
GPT-4o, Aug. 6th (temp=1)	100	100	100	48	0	69.6%
Cohere Command R+ (Aug. 2024)	100	100	100	46	0	69.2%
GPT-5.2	100	79	69	57	30	66.9%
Gemma 3 12B	100	100	61	41	25	65.4%
Gemma 3 27B	100	100	95	29	0	64.8%
Claude Haiku 4.5	100	76	73	61	0	62.1%
GPT-5.1	100	67	56	51	30	60.8%
Claude 3.5 Haiku	100	100	100	0	0	60.0%
GPT-4.1 Mini	100	100	100	0	0	60.0%
GPT-4o Mini (temp=1)	100	76	68	35	0	55.8%
Llama 3.1 70B	100	100	46	25	3	54.8%
GPT-5 Mini	92	91	36	33	8	52.1%
Mistral Large 2	100	100	46	8	0	50.8%
GPT-4.1 Nano	100	65	61	8	0	46.9%
Gemini 3 Flash (Preview)	87	85	46	8	0	45.1%
Llama 3.1 8B	100	100	25	0	0	45.0%
Gemma 3 4B	100	82	16	3	3	40.9%
Gemini 2.5 Flash Lite	100	36	20	0	0	31.2%
Llama 3.1 Nemotron 70B	81	28	22	0	0	26.1%
GPT-5 Nano	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	96	99.1%
Rocinante 12B	100	100	100	100	93	98.5%
Qwen 2.5 72B	100	100	100	100	90	98.1%
Mistral Large	100	100	100	98	92	98.0%
DeepSeek V3.2	100	100	100	100	82	96.5%
o4 Mini	100	100	100	92	90	96.5%
Llama 3.1 8B	100	100	100	99	82	96.1%
Hermes 3 405B	100	100	100	100	79	95.7%
GPT-4.1	100	100	100	100	76	95.3%
GPT-5.1	100	100	100	100	76	95.1%
GPT-4.1 Mini	100	100	100	94	81	95.0%
GPT-5.2	100	100	97	91	86	94.8%
Claude Sonnet 4.5	100	100	100	100	73	94.6%
Mistral Small 3.2 24B	100	100	100	100	69	93.9%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	67	93.3%
GPT-5	100	100	100	100	66	93.1%
Gemini 2.5 Pro	100	100	100	100	65	93.1%
Mistral Medium 3.1	100	100	100	100	65	93.0%
Gemma 3 27B	100	100	100	100	59	91.8%
Gemma 3 12B	100	100	100	94	64	91.6%
Llama 3.1 70B	100	100	100	100	54	90.8%
Ministral 3 3B	100	100	100	100	54	90.8%
Hermes 3 70B	100	100	100	88	64	90.3%
DeepSeek V3 (2025-03-24)	100	100	100	100	50	90.0%
Arcee AI: Trinity Large (Preview)	100	100	100	90	54	88.9%
Claude Sonnet 4.6	100	100	100	92	48	88.0%
Mistral Large 3	100	100	100	100	39	87.8%
GPT-4o, May 13th (temp=1)	100	100	100	88	50	87.5%
Claude 3 Haiku	100	100	100	88	50	87.5%
Claude Opus 4	100	100	100	73	62	87.2%
Claude 3.5 Sonnet	100	100	100	72	57	85.9%
GPT-4o Mini (temp=0)	100	100	100	100	27	85.4%
Gemma 3 4B	100	100	100	76	47	84.7%
Qwen 3.5 Plus (2026-02-15)	100	100	88	73	62	84.7%
Llama 3.1 Nemotron 70B	100	100	100	67	54	84.1%
WizardLM 2 8x22b	100	97	90	68	64	83.9%
Z.AI GLM 4.5	100	97	97	89	28	82.1%
Grok 4	100	100	85	78	46	81.7%
Z.AI GLM 4.7	100	100	97	74	35	81.3%
Minimax M2.5	100	100	100	57	49	81.2%
Writer: Palmyra X5	100	100	81	73	48	80.3%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
GPT-4o Mini (temp=1)	100	100	100	100	0	80.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	0	80.0%
Ministral 3B	100	100	100	100	0	80.0%
Mistral Small Creative	100	100	100	98	0	79.6%
GPT-4.1 Nano	100	100	99	56	41	79.2%
Claude Haiku 4.5	100	100	100	83	11	78.9%
DeepSeek V3.1	100	100	100	61	27	77.6%
Z.AI GLM 5	100	100	72	71	25	73.5%
Arcee AI: Trinity Mini	100	100	96	67	0	72.5%
Claude Opus 4.6	100	83	80	72	25	72.0%
Gemini 2.5 Flash	100	100	69	61	17	69.3%
Mistral Large 2	100	100	72	44	18	66.8%
Ministral 8B	100	92	72	61	3	65.5%
DeepSeek V3 (2024-12-26)	100	100	69	54	0	64.6%
Gemini 3 Flash (Preview)	100	100	62	46	11	63.7%
Z.AI GLM 4.7 Flash	100	100	97	8	0	61.2%
GPT-4o, Aug. 6th (temp=1)	100	100	89	16	0	61.0%
Claude 3.7 Sonnet	100	100	57	35	0	58.4%
Stealth: Aurora Alpha	100	100	91	0	0	58.2%
Grok 4 Fast	100	79	67	33	0	55.6%
Gemini 3 Pro (Preview)	100	86	57	23	7	54.8%
GPT-5 Mini	100	72	57	29	16	54.6%
Ministral 3 8B	100	100	50	0	0	50.0%
Z.AI GLM 4.6	80	74	62	7	0	44.6%
Ministral 3 14B	100	67	36	18	0	44.3%
Gemini 2.5 Flash Lite	100	80	35	7	0	44.3%
Claude Opus 4.5	100	54	46	18	0	43.6%
Claude Sonnet 4	100	100	0	0	0	40.0%
GPT-5 Nano	9	0	0	0	0	1.8%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	99	99.9%
Claude 3 Haiku	100	100	100	100	98	99.6%
o4 Mini	100	100	100	100	97	99.4%
Mistral NeMO	100	100	100	100	97	99.4%
Mistral Large 3	100	100	100	100	94	98.9%
Z.AI GLM 4.5	100	100	100	98	96	98.7%
o4 Mini High	100	100	100	100	92	98.4%
Llama 3.1 70B	100	100	100	100	82	96.5%
DeepSeek V3.1	100	100	100	100	81	96.1%
ByteDance Seed 1.6	100	100	100	89	89	95.6%
Ministral 8B	100	100	100	100	77	95.4%
Gemini 3.1 Pro (Preview)	100	100	100	100	76	95.3%
Mistral Small 3.2 24B	100	100	100	100	76	95.3%
GPT-4o, May 13th (temp=1)	100	100	100	93	65	91.7%
GPT-4o Mini (temp=0)	100	100	100	100	56	91.1%
Ministral 3 14B	100	100	100	100	46	89.2%
GPT-5.1	100	100	100	81	65	89.1%
DeepSeek-V2 Chat	100	100	100	92	54	89.1%
Claude Opus 4	100	100	100	100	43	88.5%
GPT-4o, Aug. 6th (temp=1)	100	100	100	94	48	88.5%
GPT-4.1	100	100	100	100	35	86.9%
Cohere Command R+ (Aug. 2024)	100	100	100	92	41	86.6%
ByteDance Seed 1.6 Flash	100	100	100	100	29	85.8%
Arcee AI: Trinity Mini	100	100	100	92	34	85.1%
DeepSeek V3 (2025-03-24)	100	100	100	67	57	84.8%
GPT-5	100	100	100	78	46	84.8%
GPT-4.1 Mini	100	100	100	100	23	84.6%
Gemini 3 Pro (Preview)	100	100	100	68	51	84.0%
Grok 4	100	100	100	74	43	83.4%
Z.AI GLM 5	100	100	96	64	54	82.7%
Gemini 2.5 Flash	100	100	79	73	61	82.6%
Gemma 3 12B	100	100	100	56	54	81.9%
Claude Sonnet 4.5	100	96	92	86	35	81.6%
Mistral Small Creative	100	100	97	64	46	81.4%
Writer: Palmyra X5	100	100	92	88	25	80.9%
Claude Sonnet 4	100	100	88	62	52	80.3%
Qwen 2.5 72B	100	100	100	100	0	80.0%
Ministral 3 3B	100	100	100	61	36	79.4%
Minimax M2.5	100	100	100	93	3	79.1%
WizardLM 2 8x22b	100	100	100	89	0	77.8%
Arcee AI: Trinity Large (Preview)	100	100	100	81	0	76.1%
Gemini 2.5 Pro	100	100	71	56	51	75.5%
Claude 3.7 Sonnet	100	100	100	38	35	74.5%
Mistral Large 2	100	100	76	56	36	73.7%
Gemini 3 Flash (Preview)	100	100	78	53	27	71.4%
Claude Opus 4.5	100	100	100	56	0	71.3%
Gemma 3 27B	100	100	73	67	11	70.2%
GPT-4o Mini (temp=1)	100	100	62	59	27	69.7%
Claude Opus 4.6	100	84	81	56	11	66.3%
Ministral 3B	100	100	64	41	25	66.0%
Claude Sonnet 4.6	92	90	84	62	0	65.8%
Z.AI GLM 4.6	100	100	69	52	0	64.3%
Gemma 3 4B	100	100	100	8	6	62.9%
Grok 4 Fast	100	83	68	46	17	62.8%
GPT-4.1 Nano	100	97	96	21	0	62.6%
Stealth: Aurora Alpha	100	100	75	30	0	61.0%
Ministral 3 8B	100	82	74	44	0	60.1%
Claude 3.5 Haiku	100	100	100	0	0	60.0%
DeepSeek V3.2	100	90	74	35	0	59.7%
Gemini 2.5 Flash Lite	100	86	59	32	0	55.5%
Llama 3.1 Nemotron 70B	88	64	44	6	0	40.1%
GPT-5 Nano	100	47	13	6	0	33.3%
GPT-5 Mini	100	30	20	0	0	29.9%
Llama 3.1 8B	65	65	11	0	0	28.3%
GPT-5.2	51	39	28	14	4	27.2%
Claude Haiku 4.5	100	29	0	0	0	25.8%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	99	99.8%
Claude 3 Haiku	100	100	100	99	97	99.2%
GPT-4.1	100	100	100	100	93	98.6%
Grok 4 Fast	100	100	100	97	89	97.2%
Rocinante 12B	100	100	100	100	80	95.9%
Claude Opus 4.6	100	99	98	98	82	95.7%
Hermes 3 405B	100	100	100	100	71	94.1%
GPT-4o Mini (temp=1)	100	100	100	100	69	93.9%
GPT-4o Mini (temp=0)	100	100	100	83	82	93.0%
Z.AI GLM 4.5	100	100	96	88	82	92.9%
DeepSeek-V2 Chat	100	100	100	100	64	92.8%
GPT-4o, May 13th (temp=1)	100	100	100	82	82	92.6%
Qwen 2.5 72B	100	100	100	100	62	92.4%
Claude 3.5 Haiku	100	100	100	100	57	91.5%
GPT-5.2	100	100	100	84	73	91.4%
GPT-4.1 Nano	100	100	100	100	56	91.1%
Gemma 3 4B	100	100	100	76	75	90.4%
Claude Sonnet 4.5	100	100	100	100	49	89.7%
GPT-5.1	100	100	100	87	56	88.6%
Ministral 3 8B	100	100	89	74	74	87.5%
Grok 4	100	100	98	77	62	87.4%
MoonshotAI: Kimi K2.5	100	100	100	73	61	86.8%
Qwen 3.5 397B A17B	100	100	100	82	51	86.6%
Z.AI GLM 5	100	100	90	88	51	85.9%
GPT-5	100	100	93	75	52	84.0%
Claude Sonnet 4.6	100	100	100	100	18	83.7%
Mistral Small Creative	100	100	100	68	50	83.6%
Qwen 3.5 Plus (2026-02-15)	100	93	92	67	65	83.3%
Llama 3.1 70B	100	100	100	100	15	83.0%
ByteDance Seed 1.6 Flash	100	100	100	100	14	82.7%
DeepSeek V3 (2025-03-24)	100	100	100	100	11	82.2%
Ministral 3 14B	100	100	98	73	35	81.1%
Z.AI GLM 4.7 Flash	100	100	97	61	46	80.7%
Mistral Small 3.2 24B	100	100	100	100	0	80.0%
Llama 3.1 8B	100	100	100	100	0	80.0%
Claude Opus 4	100	100	90	59	49	79.5%
Z.AI GLM 4.7	100	100	89	57	46	78.4%
Mistral NeMO	100	100	100	50	33	76.6%
Mistral Medium 3.1	98	93	81	73	33	75.6%
Claude Sonnet 4	100	100	78	59	36	74.5%
DeepSeek V3.2	100	98	62	62	50	74.4%
Z.AI GLM 4.6	100	100	86	82	0	73.7%
Gemma 3 12B	100	100	93	72	0	72.9%
Llama 3.1 Nemotron 70B	100	100	81	81	0	72.2%
DeepSeek V3.1	100	100	89	70	0	71.8%
Ministral 3 3B	100	100	82	69	7	71.8%
Stealth: Aurora Alpha	100	81	64	57	57	71.7%
Gemini 2.5 Pro	100	100	69	60	22	70.3%
Claude 3.5 Sonnet	100	100	86	61	0	69.3%
Mistral Large 3	100	100	91	38	15	68.8%
Arcee AI: Trinity Large (Preview)	100	100	84	54	0	67.6%
WizardLM 2 8x22b	83	72	66	56	53	66.1%
Mistral Large	100	100	100	27	0	65.4%
Mistral Large 2	100	100	98	16	0	62.8%
Claude Haiku 4.5	100	100	88	13	11	62.5%
Ministral 8B	100	100	46	41	16	60.6%
Hermes 3 70B	100	100	100	0	0	60.0%
Gemini 3 Pro (Preview)	100	69	66	32	11	55.7%
Claude Opus 4.5	100	69	49	33	20	54.2%
Gemini 2.5 Flash	93	68	62	46	1	53.9%
GPT-5 Mini	97	67	45	36	15	52.0%
Claude 3.7 Sonnet	99	84	66	0	0	49.9%
Gemini 2.5 Flash Lite	100	78	71	0	0	49.6%
Minimax M2.5	100	100	29	0	0	45.8%
Gemini 3 Flash (Preview)	100	65	64	0	0	45.7%
GPT-5 Nano	44	29	0	0	0	14.6%

genre

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Grok 4.1 Fast	100	100	100	100	100	100.0%
o4 Mini	100	100	100	91	90	96.2%
Gemini 3.1 Pro (Preview)	100	95	94	88	83	92.0%
Mistral Small 3.2 24B	100	100	100	100	54	90.9%
Mistral NeMO	100	100	100	61	59	83.9%
Claude 3 Haiku	100	100	92	72	48	82.3%
Hermes 3 405B	100	100	81	67	50	79.4%
Claude Sonnet 4.6	100	100	97	75	11	76.7%
Grok 4 Fast	100	88	73	66	52	75.7%
ByteDance Seed 1.6	100	100	100	41	22	72.6%
Ministral 3 14B	100	100	90	60	0	69.9%
DeepSeek V3 (2025-03-24)	100	100	61	44	34	67.6%
GPT-4o, May 13th (temp=0)	100	94	69	62	11	67.3%
o4 Mini High	100	92	88	56	0	67.0%
Llama 3.1 8B	100	96	76	59	0	66.2%
Writer: Palmyra X5	100	100	72	31	21	64.7%
Rocinante 12B	100	100	76	22	11	61.9%
GPT-4o Mini (temp=1)	100	100	68	41	0	61.9%
DeepSeek-V2 Chat	100	100	84	20	5	61.8%
ByteDance Seed 1.6 Flash	78	62	59	57	52	61.4%
Gemini 3 Pro (Preview)	100	71	64	60	0	58.9%
Qwen 3.5 397B A17B	76	57	56	50	49	57.4%
Mistral Large	100	76	72	18	16	56.6%
MoonshotAI: Kimi K2.5	81	62	52	47	27	53.6%
Gemini 2.5 Pro	100	100	53	11	0	52.7%
Gemma 3 27B	100	73	54	35	0	52.3%
Z.AI GLM 5	74	72	68	41	0	51.1%
DeepSeek V3.1	100	100	35	21	0	51.1%
Gemma 3 12B	76	62	56	44	0	47.8%
GPT-4o Mini (temp=0)	100	69	64	0	0	46.6%
Mistral Small Creative	100	68	59	0	0	45.3%
Mistral Medium 3.1	100	91	34	0	0	45.0%
Z.AI GLM 4.7 Flash	94	46	40	28	15	44.6%
Mistral Large 2	100	100	23	0	0	44.6%
GPT-5	100	72	40	4	0	43.1%
Gemini 2.5 Flash Lite	88	82	34	9	0	42.7%
Claude Opus 4.6	89	82	23	17	0	42.1%
Hermes 3 70B	96	96	16	0	0	41.5%
Ministral 3B	100	76	25	3	0	40.9%
Z.AI GLM 4.6	100	86	17	0	0	40.5%
Gemini 2.5 Flash	100	94	0	0	0	38.9%
DeepSeek V3 (2024-12-26)	100	50	33	0	0	36.6%
GPT-4.1	75	65	30	1	0	34.2%
DeepSeek V3.2	62	53	47	6	0	33.5%
Grok 4	81	51	34	0	0	33.1%
GPT-4.1 Nano	100	64	0	0	0	32.8%
Llama 3.1 70B	81	76	0	0	0	31.4%
Mistral Large 3	64	61	16	16	0	31.2%
Claude 3.7 Sonnet	93	38	22	0	0	30.4%
Cohere Command R+ (Aug. 2024)	100	50	0	0	0	30.0%
Minimax M2.5	69	62	7	0	0	27.8%
GPT-4o, Aug. 6th (temp=1)	65	50	16	6	0	27.4%
Qwen 2.5 72B	88	44	0	0	0	26.4%
Claude Opus 4.5	86	41	0	0	0	25.4%
Llama 3.1 Nemotron 70B	61	61	0	0	0	24.3%
Gemma 3 4B	100	8	6	0	0	22.9%
Z.AI GLM 4.7	68	29	10	0	0	21.2%
Gemini 3 Flash (Preview)	59	43	3	0	0	20.9%
Claude Sonnet 4.5	100	0	0	0	0	20.0%
GPT-4o, Aug. 6th (temp=0)	100	0	0	0	0	20.0%
GPT-5 Nano	36	36	26	0	0	19.5%
GPT-5.1	90	8	0	0	0	19.4%
GPT-4.1 Mini	33	29	16	11	6	19.0%
WizardLM 2 8x22b	94	0	0	0	0	18.9%
Claude Opus 4	36	30	20	0	0	17.2%
Ministral 3 8B	55	29	1	0	0	17.2%
Ministral 8B	42	35	0	0	0	15.4%
Qwen 3.5 Plus (2026-02-15)	47	20	0	0	0	13.5%
GPT-4o, May 13th (temp=1)	50	11	6	0	0	13.4%
GPT-5 Mini	33	30	0	0	0	12.6%
Arcee AI: Trinity Large (Preview)	56	0	0	0	0	11.3%
Z.AI GLM 4.5	23	13	13	1	0	10.0%
GPT-5.2	50	0	0	0	0	10.0%
Ministral 3 3B	48	0	0	0	0	9.6%
Claude 3.5 Sonnet	28	0	0	0	0	5.6%
Claude Haiku 4.5	23	0	0	0	0	4.6%
Claude Sonnet 4	0	0	0	0	0	0.0%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
Claude 3.5 Haiku	0	0	0	0	0	0.0%
Arcee AI: Trinity Mini	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	85	97.0%
o4 Mini High	100	100	100	100	77	95.4%
Claude 3 Haiku	100	100	100	100	74	94.8%
Mistral Small Creative	100	100	96	93	57	89.4%
Claude Sonnet 4.6	100	100	100	84	57	88.3%
Qwen 2.5 72B	100	100	100	76	51	85.6%
o4 Mini	100	100	81	79	62	84.3%
DeepSeek V3.2	100	100	96	83	36	83.0%
GPT-5.1	100	96	76	75	63	82.1%
Mistral Large 2	100	100	100	94	16	82.0%
Grok 4.1 Fast	100	100	100	88	20	81.5%
Rocinante 12B	100	100	100	69	36	81.1%
Hermes 3 70B	100	100	100	100	0	80.0%
MoonshotAI: Kimi K2.5	100	100	100	97	0	79.4%
Z.AI GLM 4.7	100	100	95	56	44	79.0%
GPT-4.1	100	96	90	65	38	77.6%
ByteDance Seed 1.6 Flash	100	88	83	83	24	75.6%
Gemini 2.5 Pro	100	100	76	62	39	75.3%
Gemma 3 12B	100	79	79	67	49	74.5%
DeepSeek-V2 Chat	100	100	100	35	34	73.8%
Mistral Large 3	100	100	100	43	18	72.3%
Ministral 3 14B	100	81	73	64	43	72.1%
Ministral 3 8B	98	83	78	71	20	70.0%
Hermes 3 405B	100	98	84	59	0	68.2%
Mistral Small 3.2 24B	100	100	100	39	0	67.8%
Mistral Large	100	100	89	46	0	67.0%
Qwen 3.5 397B A17B	93	89	81	67	0	66.0%
Cohere Command R+ (Aug. 2024)	100	100	64	49	0	62.6%
Claude Opus 4.6	97	85	81	50	0	62.5%
Z.AI GLM 4.7 Flash	100	84	60	39	30	62.3%
Gemini 3 Flash (Preview)	98	73	68	68	0	61.6%
Gemini 3.1 Pro (Preview)	100	81	70	25	23	59.8%
GPT-5	100	87	79	32	0	59.7%
Gemini 2.5 Flash	100	100	37	35	23	59.0%
Gemini 2.5 Flash Lite	100	78	53	45	0	55.0%
Z.AI GLM 5	100	100	74	0	0	54.8%
Claude 3.7 Sonnet	72	55	51	50	42	54.0%
Ministral 8B	100	65	65	28	6	52.9%
Minimax M2.5	100	100	43	13	0	51.2%
GPT-4o, Aug. 6th (temp=1)	100	100	35	18	0	50.6%
Claude Opus 4.5	100	100	53	0	0	50.6%
Arcee AI: Trinity Large (Preview)	86	80	41	35	8	49.9%
Ministral 3 3B	100	70	61	18	0	49.8%
Qwen 3.5 Plus (2026-02-15)	80	79	77	0	0	47.3%
Gemma 3 27B	100	85	47	3	0	47.1%
Grok 4 Fast	100	98	32	0	0	46.1%
Claude Opus 4	100	82	43	5	0	46.1%
Claude Haiku 4.5	100	82	44	0	0	45.0%
Ministral 3B	96	69	22	18	0	41.2%
Z.AI GLM 4.6	81	62	46	17	0	41.0%
GPT-4o Mini (temp=1)	100	89	13	0	0	40.4%
Llama 3.1 70B	100	96	6	0	0	40.3%
DeepSeek V3.1	100	57	27	17	0	40.2%
Gemini 3 Pro (Preview)	77	71	30	18	0	39.2%
GPT-4o, May 13th (temp=1)	78	56	49	11	0	38.7%
Claude Sonnet 4.5	100	84	9	0	0	38.7%
DeepSeek V3 (2024-12-26)	83	44	36	22	0	37.2%
GPT-5.2	62	59	35	24	0	36.1%
Claude 3.5 Sonnet	74	52	50	0	0	35.2%
Mistral NeMO	78	56	38	3	0	34.8%
GPT-4.1 Nano	74	68	25	0	0	33.5%
Claude Sonnet 4	89	36	0	0	0	25.1%
GPT-4o Mini (temp=0)	53	41	25	0	0	23.8%
Grok 4	100	10	7	0	0	23.4%
Gemma 3 4B	76	33	0	0	0	21.9%
GPT-5 Mini	73	36	0	0	0	21.8%
Arcee AI: Trinity Mini	41	38	21	9	0	21.8%
Z.AI GLM 4.5	88	13	0	0	0	20.1%
Llama 3.1 Nemotron 70B	100	0	0	0	0	20.0%
GPT-4.1 Mini	78	15	0	0	0	18.5%
GPT-4o, Aug. 6th (temp=0)	68	5	3	1	0	15.4%
ByteDance Seed 1.6	44	31	1	0	0	15.2%
GPT-4o, May 13th (temp=0)	61	7	0	0	0	13.6%
Llama 3.1 8B	62	1	0	0	0	12.6%
WizardLM 2 8x22b	23	18	14	5	0	11.9%
GPT-5 Nano	33	24	0	0	0	11.5%
Claude 3.5 Haiku	18	0	0	0	0	3.7%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude 3 Haiku	100	100	100	100	100	100.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	92	98.4%
Claude 3.7 Sonnet	100	100	100	100	85	97.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	84	96.8%
DeepSeek V3 (2024-12-26)	100	100	100	98	84	96.4%
Gemini 3.1 Pro (Preview)	100	100	100	100	76	95.3%
Mistral Large 2	100	100	100	100	76	95.3%
Grok 4.1 Fast	100	100	100	100	71	94.2%
Mistral Medium 3.1	100	100	100	100	68	93.6%
GPT-4o, May 13th (temp=0)	100	100	100	100	65	93.1%
o4 Mini High	100	100	92	90	80	92.4%
GPT-4o Mini (temp=0)	100	100	100	100	46	89.2%
ByteDance Seed 1.6	100	100	100	100	36	87.3%
MoonshotAI: Kimi K2.5	100	100	100	100	31	86.2%
Claude 3.5 Sonnet	100	100	100	100	31	86.2%
Mistral Large	100	100	100	81	47	85.6%
DeepSeek V3 (2025-03-24)	100	100	100	89	36	85.1%
Gemma 3 27B	100	88	86	79	69	84.3%
Mistral Small 3.2 24B	100	100	100	82	36	83.6%
Rocinante 12B	100	100	100	69	48	83.5%
Gemini 2.5 Pro	100	100	100	91	25	83.2%
ByteDance Seed 1.6 Flash	100	100	83	82	47	82.6%
Ministral 3 3B	100	96	88	74	54	82.2%
Ministral 8B	100	100	85	64	56	81.0%
Hermes 3 70B	100	100	84	69	49	80.4%
Gemini 3 Pro (Preview)	100	92	88	85	36	80.3%
Hermes 3 405B	100	100	100	100	0	80.0%
Ministral 3 8B	100	100	100	85	14	79.7%
Qwen 2.5 72B	100	100	100	79	7	77.1%
Minimax M2.5	100	100	100	82	3	77.1%
Mistral Small Creative	100	86	82	59	57	77.0%
Ministral 3 14B	100	100	92	57	35	76.8%
Z.AI GLM 4.7 Flash	100	100	100	55	25	76.0%
Gemma 3 12B	100	100	100	50	30	75.9%
GPT-5.1	100	87	77	66	41	74.2%
Claude Opus 4	100	100	100	47	18	73.1%
o4 Mini	100	94	87	49	32	72.4%
GPT-4o, May 13th (temp=1)	100	74	67	65	56	72.4%
Gemini 2.5 Flash Lite	100	100	85	71	0	71.1%
Llama 3.1 8B	100	96	94	62	0	70.5%
Claude Sonnet 4	100	100	72	67	9	69.5%
DeepSeek-V2 Chat	100	100	100	46	0	69.2%
Grok 4 Fast	100	100	67	48	29	68.7%
Mistral Large 3	100	100	100	41	0	68.3%
Z.AI GLM 4.5	100	100	81	39	21	68.0%
DeepSeek V3.2	100	100	95	28	17	68.0%
Z.AI GLM 5	100	100	59	50	27	67.2%
Z.AI GLM 4.7	100	100	89	38	8	67.0%
Claude Sonnet 4.6	100	100	100	35	0	66.9%
Claude Opus 4.6	100	100	81	51	0	66.4%
Gemini 3 Flash (Preview)	85	76	75	50	41	65.6%
Qwen 3.5 397B A17B	100	100	80	46	0	65.1%
Arcee AI: Trinity Large (Preview)	100	100	100	17	3	63.9%
Claude Opus 4.5	100	100	60	51	0	62.2%
GPT-4o, Aug. 6th (temp=1)	100	100	59	50	0	61.8%
GPT-4.1	100	78	47	43	40	61.5%
Qwen 3.5 Plus (2026-02-15)	100	78	67	61	0	61.0%
Mistral NeMO	100	100	100	0	0	60.0%
GPT-5	100	89	79	25	4	59.5%
Claude 3.5 Haiku	100	100	79	18	0	59.4%
Grok 4	89	80	65	60	0	58.6%
Ministral 3B	100	100	72	14	0	57.1%
WizardLM 2 8x22b	100	100	50	35	0	56.9%
Arcee AI: Trinity Mini	100	97	85	0	0	56.4%
Llama 3.1 Nemotron 70B	100	92	48	41	0	56.2%
Gemma 3 4B	100	100	56	23	0	55.7%
Z.AI GLM 4.6	100	68	59	47	0	54.9%
DeepSeek V3.1	68	63	60	27	0	43.7%
Llama 3.1 70B	100	100	16	0	0	43.2%
Gemini 2.5 Flash	58	55	40	36	13	40.5%
GPT-4.1 Nano	100	48	46	0	0	38.8%
GPT-4o Mini (temp=1)	100	31	29	0	0	32.0%
Claude Sonnet 4.5	71	40	21	8	5	28.9%
Claude Haiku 4.5	61	0	0	0	0	12.1%
GPT-4.1 Mini	41	18	0	0	0	11.9%
GPT-5.2	31	22	0	0	0	10.6%
GPT-5 Mini	38	0	0	0	0	7.6%
GPT-5 Nano	11	1	0	0	0	2.4%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	99	99.8%
GPT-4o Mini (temp=1)	100	100	100	99	96	98.9%
Ministral 3 8B	100	100	100	100	93	98.6%
Qwen 2.5 72B	100	100	100	100	90	98.1%
Hermes 3 405B	100	100	100	100	89	97.8%
Claude 3 Haiku	100	100	100	100	86	97.2%
o4 Mini	100	100	100	100	82	96.5%
GPT-4.1 Mini	100	100	100	92	89	96.2%
Ministral 3B	100	100	100	92	74	93.2%
DeepSeek V3 (2025-03-24)	100	100	100	100	64	92.8%
Arcee AI: Trinity Mini	100	100	100	100	64	92.8%
Mistral NeMO	100	100	98	83	79	92.0%
Claude 3.5 Haiku	100	100	100	100	57	91.5%
Writer: Palmyra X5	100	100	100	100	54	90.8%
o4 Mini High	100	100	100	76	74	90.1%
Llama 3.1 70B	100	100	90	82	76	89.9%
Mistral Small 3.2 24B	100	100	100	100	47	89.4%
WizardLM 2 8x22b	100	100	100	82	64	89.2%
GPT-5.1	100	100	94	75	75	88.8%
Hermes 3 70B	100	100	93	89	50	86.4%
Grok 4	100	100	100	75	55	86.1%
Gemma 3 4B	100	100	100	100	29	85.8%
Gemini 3 Pro (Preview)	100	100	100	74	54	85.6%
DeepSeek V3 (2024-12-26)	100	100	90	71	64	85.0%
Rocinante 12B	100	100	100	100	11	82.2%
GPT-4o, Aug. 6th (temp=0)	100	100	100	90	18	81.8%
GPT-4.1	100	100	100	84	20	80.9%
Gemma 3 27B	100	100	72	65	64	80.2%
DeepSeek V3.1	100	100	100	65	33	79.6%
ByteDance Seed 1.6	100	100	100	76	18	79.0%
Claude Opus 4	100	100	100	62	31	78.6%
Llama 3.1 Nemotron 70B	100	100	100	67	22	77.7%
Minimax M2.5	100	100	65	64	56	76.9%
Z.AI GLM 4.7	88	86	84	68	57	76.7%
Mistral Large	100	100	100	41	31	74.5%
Gemma 3 12B	100	100	69	68	27	72.9%
Cohere Command R+ (Aug. 2024)	100	100	73	72	18	72.7%
Z.AI GLM 4.7 Flash	100	100	100	62	0	72.4%
GPT-5 Mini	100	91	85	71	14	72.1%
GPT-5	100	100	85	66	9	72.0%
DeepSeek-V2 Chat	100	100	90	36	31	71.6%
Ministral 3 3B	100	100	100	34	17	70.1%
Mistral Large 3	100	100	100	46	0	69.2%
GPT-4.1 Nano	100	98	84	57	0	67.9%
Mistral Large 2	100	94	67	48	25	66.8%
DeepSeek V3.2	100	85	62	61	25	66.5%
GPT-4o, May 13th (temp=1)	100	100	100	16	0	63.2%
Claude 3.7 Sonnet	100	100	65	49	0	62.8%
Z.AI GLM 4.6	100	82	76	50	0	61.8%
Qwen 3.5 Plus (2026-02-15)	100	100	47	31	27	61.1%
Ministral 3 14B	100	100	100	5	0	61.0%
Gemini 2.5 Flash	100	100	57	47	0	60.9%
Mistral Medium 3.1	100	82	61	51	6	60.0%
Ministral 8B	100	94	89	16	0	59.9%
Claude Sonnet 4.6	100	100	57	38	0	59.1%
GPT-5.2	85	83	68	54	0	58.0%
Z.AI GLM 4.5	100	54	50	44	39	57.3%
Gemini 3 Flash (Preview)	100	83	54	27	17	56.2%
Gemini 2.5 Flash Lite	100	93	76	6	0	55.1%
Claude Opus 4.6	100	81	41	40	0	52.4%
Claude Haiku 4.5	100	98	57	0	0	51.1%
Gemini 2.5 Pro	89	72	53	39	0	50.5%
Claude Opus 4.5	100	100	51	0	0	50.3%
Claude Sonnet 4.5	83	57	43	43	18	49.0%
Llama 3.1 8B	100	74	45	25	0	48.8%
Arcee AI: Trinity Large (Preview)	100	56	35	31	18	47.9%
MoonshotAI: Kimi K2.5	93	73	61	0	0	45.3%
Claude Sonnet 4	100	100	0	0	0	40.0%
Grok 4 Fast	100	40	29	25	0	38.7%
Z.AI GLM 5	94	41	31	5	0	34.2%
Claude 3.5 Sonnet	100	36	28	0	0	32.9%
Stealth: Aurora Alpha	100	40	0	0	0	27.9%
GPT-4o, Aug. 6th (temp=1)	98	0	0	0	0	19.6%
GPT-5 Nano	43	10	0	0	0	10.6%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	97	99.4%
Hermes 3 405B	100	100	100	100	96	99.1%
Ministral 3 8B	100	100	100	100	93	98.6%
Ministral 8B	100	100	100	100	93	98.5%
DeepSeek V3 (2025-03-24)	100	100	100	93	90	96.7%
Claude Opus 4	100	100	100	100	82	96.5%
GPT-4o Mini (temp=1)	100	100	100	100	81	96.1%
Ministral 3 3B	100	100	100	88	88	95.0%
Claude 3.7 Sonnet	100	100	93	90	85	93.6%
DeepSeek V3.2	100	100	100	94	69	92.8%
Grok 4.1 Fast	100	100	96	89	73	91.6%
Claude 3.5 Sonnet	100	92	89	88	86	90.9%
Grok 4 Fast	100	100	100	92	62	90.7%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	50	90.0%
Mistral Small Creative	100	100	100	88	54	88.3%
Hermes 3 70B	100	100	100	100	34	86.7%
Gemini 2.5 Flash	100	100	98	97	35	86.2%
Claude Sonnet 4	100	100	100	82	33	82.9%
DeepSeek V3 (2024-12-26)	100	100	92	71	49	82.2%
Mistral Large	100	100	88	65	57	82.0%
Claude 3 Haiku	100	100	100	99	11	82.0%
Gemma 3 12B	100	100	74	68	62	80.9%
Grok 4	100	100	100	60	44	80.7%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Writer: Palmyra X5	100	100	100	100	0	80.0%
Arcee AI: Trinity Mini	100	97	96	76	25	78.8%
Ministral 3 14B	100	100	100	93	0	78.6%
Gemini 2.5 Pro	100	100	98	90	0	77.8%
Qwen 3.5 397B A17B	100	100	76	75	35	77.4%
Llama 3.1 70B	100	100	94	88	0	76.4%
Cohere Command R+ (Aug. 2024)	100	100	100	50	31	76.2%
o4 Mini	100	100	87	59	31	75.2%
GPT-4o, Aug. 6th (temp=0)	100	100	93	69	11	74.7%
ByteDance Seed 1.6 Flash	100	100	79	77	13	73.7%
Z.AI GLM 4.5	100	100	80	65	18	72.7%
Claude Sonnet 4.5	100	100	100	52	8	72.1%
Gemma 3 27B	100	100	100	52	7	71.8%
DeepSeek-V2 Chat	100	100	79	71	7	71.3%
Qwen 2.5 72B	100	100	78	56	23	71.2%
o4 Mini High	100	90	82	49	28	69.6%
Gemini 3 Flash (Preview)	100	86	80	47	33	69.1%
Mistral Large 2	100	94	56	54	39	68.6%
Z.AI GLM 4.6	100	92	90	60	0	68.3%
Z.AI GLM 4.7	95	93	72	68	4	66.6%
Claude Sonnet 4.6	100	97	88	44	0	65.6%
Claude Opus 4.6	100	90	56	40	36	64.6%
Ministral 3B	90	84	79	46	22	64.2%
Gemini 2.5 Flash Lite	90	79	76	50	25	64.1%
Gemma 3 4B	100	98	97	25	0	63.9%
GPT-5.1	100	85	44	41	38	61.7%
WizardLM 2 8x22b	100	92	69	44	0	61.0%
GPT-4.1 Nano	100	100	97	0	0	59.4%
Llama 3.1 8B	100	79	58	33	22	58.4%
GPT-4.1	100	92	49	29	18	57.6%
GPT-4o, May 13th (temp=0)	100	57	54	38	35	56.8%
Z.AI GLM 4.7 Flash	79	65	53	51	30	55.6%
Claude Haiku 4.5	100	68	61	48	0	55.3%
Minimax M2.5	100	98	35	25	18	55.2%
Qwen 3.5 Plus (2026-02-15)	100	73	64	38	0	55.0%
Claude Opus 4.5	100	100	71	0	0	54.1%
MoonshotAI: Kimi K2.5	100	100	69	0	0	53.9%
ByteDance Seed 1.6	100	52	44	31	27	50.7%
DeepSeek V3.1	100	90	50	13	0	50.7%
Mistral Large 3	100	67	36	0	0	40.6%
GPT-5	66	51	43	22	0	36.2%
GPT-5.2	60	33	30	24	17	32.8%
Z.AI GLM 5	83	75	1	0	0	31.9%
Llama 3.1 Nemotron 70B	82	74	0	0	0	31.3%
GPT-4.1 Mini	59	46	41	0	0	29.2%
GPT-5 Mini	17	0	0	0	0	3.5%
Stealth: Aurora Alpha	1	0	0	0	0	0.3%
GPT-5 Nano	0	0	0	0	0	0.0%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
o4 Mini High	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
Mistral NeMO	100	100	100	100	98	99.6%
Claude 3 Haiku	100	100	100	100	97	99.4%
Grok 4.1 Fast	100	100	100	98	95	98.6%
Qwen 2.5 72B	100	100	100	97	95	98.5%
GPT-4o, May 13th (temp=0)	100	100	100	100	88	97.7%
Cohere Command R+ (Aug. 2024)	100	100	100	96	89	96.9%
Rocinante 12B	100	100	100	100	82	96.5%
GPT-4.1 Mini	100	100	100	100	81	96.1%
Qwen 3.5 397B A17B	100	100	100	100	80	95.9%
Gemini 3.1 Pro (Preview)	100	100	100	100	76	95.3%
WizardLM 2 8x22b	100	100	100	100	76	95.3%
Hermes 3 70B	100	100	100	97	80	95.3%
Arcee AI: Trinity Mini	100	100	100	92	84	95.2%
ByteDance Seed 1.6	100	100	100	100	75	95.1%
ByteDance Seed 1.6 Flash	100	100	100	100	75	95.0%
Llama 3.1 Nemotron 70B	100	100	100	86	84	94.0%
Ministral 3 14B	100	100	100	91	69	92.1%
Claude Sonnet 4.5	100	100	100	82	78	91.8%
Z.AI GLM 4.7	100	100	100	93	62	91.0%
GPT-5	100	100	92	89	65	89.3%
GPT-4o, May 13th (temp=1)	100	100	100	90	53	88.6%
GPT-4.1	100	96	90	86	69	88.3%
DeepSeek V3.2	100	100	94	82	62	87.5%
Mistral Large 3	100	99	98	92	47	87.2%
GPT-4o Mini (temp=1)	100	100	100	100	29	85.8%
DeepSeek-V2 Chat	100	100	84	81	59	84.8%
GPT-4o, Aug. 6th (temp=0)	100	100	100	62	49	82.2%
Mistral Large	100	100	73	68	67	81.6%
Gemini 2.5 Pro	100	100	86	72	50	81.6%
Gemma 3 4B	100	100	98	73	36	81.5%
DeepSeek V3.1	100	100	73	67	61	80.0%
Gemma 3 12B	100	100	88	71	38	79.4%
MoonshotAI: Kimi K2.5	92	84	81	74	61	78.3%
Ministral 3 8B	100	100	100	79	0	75.7%
GPT-4o, Aug. 6th (temp=1)	100	100	100	56	16	74.3%
Mistral Small 3.2 24B	100	100	97	72	0	73.8%
Mistral Medium 3.1	100	100	100	42	23	73.1%
Grok 4 Fast	100	100	93	65	0	71.7%
Mistral Large 2	100	100	83	73	0	71.4%
DeepSeek V3 (2024-12-26)	100	100	90	61	0	70.2%
Mistral Small Creative	100	100	75	56	18	69.8%
Claude Haiku 4.5	100	100	86	59	0	69.0%
Grok 4	100	100	88	45	6	67.6%
DeepSeek V3 (2025-03-24)	100	67	61	59	48	66.9%
Gemini 3 Pro (Preview)	100	100	81	34	17	66.4%
GPT-4.1 Nano	100	100	62	35	35	66.3%
Claude 3.7 Sonnet	100	100	52	41	30	64.6%
Z.AI GLM 4.5	100	100	98	20	0	63.6%
Claude Opus 4	100	89	88	25	14	63.2%
Z.AI GLM 4.7 Flash	86	74	70	46	27	60.7%
Qwen 3.5 Plus (2026-02-15)	86	83	73	56	0	59.8%
GPT-5 Mini	74	70	61	57	25	57.5%
GPT-4o Mini (temp=0)	100	84	53	29	21	57.3%
Llama 3.1 70B	100	100	39	36	11	57.3%
Gemma 3 27B	100	100	61	20	0	56.2%
Ministral 8B	100	91	67	17	0	54.9%
GPT-5.1	100	82	44	40	2	53.7%
Z.AI GLM 5	100	68	62	35	0	53.1%
Gemini 2.5 Flash Lite	100	72	64	7	0	48.6%
Claude Sonnet 4	100	75	52	14	0	48.2%
Claude Sonnet 4.6	100	65	59	11	0	47.1%
Claude Opus 4.6	93	57	56	18	0	44.9%
Arcee AI: Trinity Large (Preview)	82	74	42	17	3	43.8%
GPT-5.2	100	76	21	18	0	43.0%
Minimax M2.5	100	73	28	0	0	40.2%
Llama 3.1 8B	100	61	25	5	0	38.1%
Claude Opus 4.5	100	69	13	1	0	36.6%
Gemini 3 Flash (Preview)	97	59	24	0	0	35.8%
Z.AI GLM 4.6	86	46	17	16	3	33.4%
Claude 3.5 Haiku	100	54	0	0	0	30.8%
Gemini 2.5 Flash	83	38	25	0	0	29.3%
Claude 3.5 Sonnet	100	41	0	0	0	28.3%
GPT-5 Nano	47	0	0	0	0	9.3%
Stealth: Aurora Alpha	45	0	0	0	0	8.9%

Novelcrafter Default Prompt

▼

Fantasy: entering an ancient ruin

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	84	96.8%
MoonshotAI: Kimi K2.5	100	100	100	100	81	96.1%
GPT-4o, May 13th (temp=0)	100	100	100	88	82	94.0%
o4 Mini	100	100	90	82	66	87.7%
Qwen 3.5 397B A17B	100	100	100	100	37	87.5%
GPT-4o Mini (temp=1)	100	100	100	69	62	86.3%
o4 Mini High	100	100	100	89	32	84.2%
DeepSeek V3 (2025-03-24)	100	100	99	97	3	79.7%
Mistral Medium 3.1	100	100	100	55	23	75.6%
Grok 4 Fast	100	100	68	67	41	75.3%
Llama 3.1 70B	100	98	82	67	22	73.8%
DeepSeek-V2 Chat	100	99	94	47	23	72.9%
GPT-4.1	100	100	86	47	23	71.3%
Mistral Large	100	83	80	65	22	70.0%
GPT-4o, Aug. 6th (temp=1)	100	100	62	52	33	69.4%
Hermes 3 405B	100	100	100	39	0	67.8%
Z.AI GLM 5	100	100	78	53	0	66.0%
GPT-5.1	100	98	76	56	0	65.9%
Gemini 3 Flash (Preview)	100	89	81	44	0	62.6%
GPT-5	100	86	57	36	32	62.1%
Mistral Small Creative	100	93	62	52	0	61.5%
GPT-4o, Aug. 6th (temp=0)	100	100	65	29	8	60.6%
Ministral 3 14B	100	88	59	36	18	60.4%
Qwen 2.5 72B	100	76	69	54	0	59.9%
Mistral Large 3	98	84	61	56	0	59.8%
Claude Opus 4	100	90	72	35	0	59.2%
Mistral NeMO	100	100	96	0	0	59.1%
Gemini 2.5 Pro	100	100	82	3	0	57.1%
Hermes 3 70B	100	69	57	57	0	56.8%
ByteDance Seed 1.6 Flash	100	100	59	25	0	56.8%
WizardLM 2 8x22b	100	100	79	0	0	55.8%
ByteDance Seed 1.6	100	100	61	0	0	52.1%
Claude Sonnet 4.6	100	98	56	6	0	51.9%
Claude Opus 4.6	91	65	55	42	0	50.7%
Grok 4	95	91	32	30	0	49.6%
GPT-4.1 Mini	100	100	41	6	0	49.4%
Writer: Palmyra X5	100	100	31	15	0	49.2%
DeepSeek V3 (2024-12-26)	100	56	52	36	0	48.9%
Arcee AI: Trinity Mini	100	100	44	0	0	48.7%
Mistral Large 2	100	94	49	0	0	48.5%
Gemini 3 Pro (Preview)	100	72	35	32	0	47.9%
Claude Opus 4.5	100	100	35	3	0	47.5%
GPT-4o Mini (temp=0)	100	79	47	11	0	47.4%
Cohere Command R+ (Aug. 2024)	100	100	31	0	0	46.2%
Gemma 3 12B	100	80	47	0	0	45.4%
Gemini 2.5 Flash	72	66	64	21	0	44.4%
Z.AI GLM 4.6	100	59	41	16	0	43.3%
Gemma 3 4B	100	64	49	3	0	43.1%
Mistral Small 3.2 24B	100	77	28	0	0	41.1%
Qwen 3.5 Plus (2026-02-15)	86	86	30	1	0	40.4%
GPT-4o, May 13th (temp=1)	81	71	46	0	0	39.4%
Z.AI GLM 4.7	87	36	31	26	17	39.4%
GPT-4.1 Nano	71	68	56	0	0	38.9%
Gemini 2.5 Flash Lite	100	72	16	0	0	37.6%
Ministral 3B	100	86	0	0	0	37.2%
Arcee AI: Trinity Large (Preview)	100	74	0	0	0	34.8%
Z.AI GLM 4.7 Flash	81	54	36	0	0	34.2%
Claude Haiku 4.5	100	35	21	8	3	33.3%
Claude Sonnet 4.5	100	36	25	1	0	32.4%
Ministral 3 3B	72	64	18	7	0	32.2%
Llama 3.1 8B	76	74	0	0	0	30.1%
Gemma 3 27B	73	59	18	0	0	30.1%
Ministral 3 8B	81	54	16	0	0	30.1%
Claude 3 Haiku	100	43	3	0	0	29.2%
Llama 3.1 Nemotron 70B	67	64	11	0	0	28.3%
GPT-5.2	51	49	39	0	0	27.8%
Claude Sonnet 4	100	0	0	0	0	20.0%
Claude 3.5 Haiku	100	0	0	0	0	20.0%
Ministral 8B	36	31	7	0	0	14.9%
Minimax M2.5	73	0	0	0	0	14.6%
DeepSeek V3.2	54	13	5	0	0	14.4%
Claude 3.7 Sonnet	64	1	0	0	0	12.9%
GPT-5 Mini	29	0	0	0	0	5.8%
DeepSeek V3.1	25	0	0	0	0	5.0%
Stealth: Aurora Alpha	15	0	0	0	0	3.0%
GPT-5 Nano	12	0	0	0	0	2.5%
Claude 3.5 Sonnet	7	0	0	0	0	1.4%
Z.AI GLM 4.5	0	0	0	0	0	0.0%

▼

Horror: alone in an eerie place at night

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	94	98.9%
Qwen 3.5 397B A17B	100	100	100	100	92	98.4%
Claude Sonnet 4.6	100	100	100	98	90	97.7%
GPT-5.1	100	100	100	100	80	96.0%
Claude Opus 4.6	100	100	100	100	76	95.3%
o4 Mini	100	100	100	100	71	94.3%
Mistral Medium 3.1	100	100	100	100	68	93.6%
Mistral Large 3	100	100	100	100	62	92.5%
Hermes 3 405B	100	100	100	100	62	92.5%
MoonshotAI: Kimi K2.5	100	100	100	87	74	92.2%
DeepSeek V3.2	100	100	100	92	68	91.9%
Ministral 3 14B	100	100	96	90	68	90.8%
Mistral Small Creative	100	100	100	100	54	90.8%
GPT-5	100	100	100	74	55	85.8%
Grok 4	100	100	100	76	49	85.1%
DeepSeek V3 (2024-12-26)	100	100	100	68	56	84.8%
WizardLM 2 8x22b	100	100	97	76	50	84.6%
Claude Opus 4	100	100	100	74	49	84.6%
ByteDance Seed 1.6 Flash	100	100	100	78	18	79.2%
Hermes 3 70B	100	100	100	94	0	78.9%
Llama 3.1 8B	100	100	86	74	31	78.2%
Mistral Large 2	100	100	92	76	16	76.9%
Gemini 3 Flash (Preview)	100	100	85	71	28	76.7%
Qwen 2.5 72B	100	100	82	57	38	75.4%
GPT-4o Mini (temp=0)	100	100	100	46	30	75.1%
Ministral 3 3B	100	100	67	54	54	74.9%
Claude 3 Haiku	100	100	100	69	0	73.9%
GPT-4.1	88	78	74	65	56	72.2%
Gemini 2.5 Flash Lite	100	100	82	73	0	70.9%
Ministral 8B	100	100	81	59	15	70.9%
Gemini 2.5 Pro	100	100	100	47	0	69.5%
GPT-5.2	100	98	62	49	38	69.4%
Mistral Large	100	82	69	64	31	69.3%
Z.AI GLM 5	100	100	73	63	3	67.8%
Cohere Command R+ (Aug. 2024)	100	100	82	56	0	67.6%
Claude 3.7 Sonnet	100	92	76	66	0	66.7%
Z.AI GLM 4.7	100	100	71	60	0	66.1%
Qwen 3.5 Plus (2026-02-15)	100	84	61	46	39	66.0%
GPT-4o, May 13th (temp=1)	90	89	75	58	11	64.7%
Gemma 3 4B	100	100	94	23	0	63.6%
Mistral NeMO	100	85	72	59	0	63.2%
DeepSeek V3 (2025-03-24)	100	100	86	22	0	61.5%
Gemma 3 27B	100	100	92	15	0	61.3%
Gemini 3 Pro (Preview)	100	81	61	34	27	60.4%
Gemini 2.5 Flash	100	100	59	41	0	60.1%
GPT-4o, May 13th (temp=0)	100	98	73	18	11	60.0%
Mistral Small 3.2 24B	100	100	100	0	0	60.0%
Minimax M2.5	100	90	57	47	0	58.8%
Ministral 3B	82	72	69	67	0	58.1%
DeepSeek-V2 Chat	100	97	86	0	0	56.5%
Claude Sonnet 4.5	96	88	36	23	22	53.1%
Z.AI GLM 4.5	100	81	56	14	0	50.2%
Claude Opus 4.5	99	99	46	3	0	49.6%
Gemma 3 12B	100	88	57	0	0	49.0%
Z.AI GLM 4.7 Flash	99	54	48	43	0	48.8%
GPT-4o Mini (temp=1)	84	60	40	38	18	48.0%
Ministral 3 8B	100	100	39	0	0	47.8%
DeepSeek V3.1	100	100	38	0	0	47.5%
ByteDance Seed 1.6	100	56	41	38	0	47.1%
GPT-5 Mini	84	81	69	0	0	46.8%
Claude Sonnet 4	100	100	27	0	0	45.4%
GPT-4.1 Mini	100	61	40	11	0	42.3%
Z.AI GLM 4.6	56	55	36	31	28	41.3%
GPT-4o, Aug. 6th (temp=1)	76	72	29	23	0	40.1%
Claude Haiku 4.5	100	68	17	15	0	40.0%
Arcee AI: Trinity Large (Preview)	100	86	9	0	0	39.0%
Claude 3.5 Sonnet	100	31	0	0	0	26.2%
Llama 3.1 70B	48	44	34	0	0	25.1%
Claude 3.5 Haiku	100	0	0	0	0	20.0%
GPT-4.1 Nano	41	33	17	5	0	19.2%
GPT-4o, Aug. 6th (temp=0)	35	30	17	0	0	16.2%
Llama 3.1 Nemotron 70B	36	0	0	0	0	7.3%
Arcee AI: Trinity Mini	33	0	0	0	0	6.6%
Stealth: Aurora Alpha	0	0	0	0	0	0.0%
GPT-5 Nano	0	0	0	0	0	0.0%

▼

Literary fiction: old friends reunite

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Rocinante 12B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	99	99.8%
Grok 4 Fast	100	100	100	100	98	99.6%
Writer: Palmyra X5	100	100	100	99	97	99.2%
Mistral NeMO	100	100	100	93	93	97.1%
Gemini 3 Pro (Preview)	100	100	100	100	82	96.3%
o4 Mini High	100	100	100	94	87	96.2%
DeepSeek V3 (2025-03-24)	100	100	100	93	88	96.1%
Ministral 3B	100	100	100	100	76	95.3%
Ministral 3 8B	100	100	100	99	76	95.1%
GPT-4o, May 13th (temp=0)	100	100	100	100	73	94.6%
Ministral 8B	100	100	100	98	74	94.5%
MoonshotAI: Kimi K2.5	100	100	100	91	76	93.5%
Claude Opus 4	100	100	100	100	64	92.8%
Ministral 3 14B	100	100	100	100	61	92.1%
Minimax M2.5	100	100	100	100	59	91.8%
Hermes 3 405B	100	100	100	82	76	91.8%
o4 Mini	100	100	100	100	59	91.7%
Z.AI GLM 4.6	100	100	100	89	57	89.3%
Mistral Large 3	100	100	100	86	56	88.3%
Mistral Small 3.2 24B	100	100	100	100	32	86.5%
Hermes 3 70B	100	100	100	96	36	86.4%
DeepSeek-V2 Chat	100	100	100	100	31	86.2%
Z.AI GLM 4.7	100	100	100	80	51	86.2%
ByteDance Seed 1.6	100	100	100	100	22	84.4%
Mistral Medium 3.1	100	100	100	59	57	83.3%
Gemini 3 Flash (Preview)	100	100	87	86	41	82.8%
GPT-5.2	100	100	89	61	59	81.9%
Claude 3.7 Sonnet	100	100	100	57	52	81.9%
Cohere Command R+ (Aug. 2024)	100	100	100	100	0	80.0%
Gemma 3 4B	100	100	100	100	0	80.0%
Qwen 2.5 72B	100	100	99	94	0	78.7%
DeepSeek V3.2	100	88	88	85	33	78.6%
Claude Sonnet 4	100	100	100	92	0	78.4%
Z.AI GLM 4.5	100	100	98	67	27	78.3%
GPT-4o Mini (temp=0)	100	100	99	64	25	77.6%
DeepSeek V3.1	100	100	82	82	18	76.3%
DeepSeek V3 (2024-12-26)	100	100	100	81	0	76.1%
GPT-5.1	100	100	100	67	8	75.0%
Claude Sonnet 4.5	100	100	100	41	21	72.4%
GPT-4o Mini (temp=1)	100	100	79	43	39	72.1%
Claude Opus 4.6	100	100	72	48	36	71.2%
Gemini 2.5 Pro	100	100	81	60	11	70.3%
Arcee AI: Trinity Large (Preview)	100	100	89	39	18	69.3%
Mistral Large	100	100	71	64	11	69.1%
GPT-5	99	96	61	55	32	68.8%
Z.AI GLM 4.7 Flash	100	82	75	46	40	68.5%
Z.AI GLM 5	100	100	100	38	0	67.6%
Arcee AI: Trinity Mini	100	100	100	34	0	66.7%
Gemma 3 27B	100	100	62	48	0	62.0%
GPT-4o, May 13th (temp=1)	100	100	100	8	0	61.7%
Claude Opus 4.5	100	86	62	47	1	59.2%
Claude 3 Haiku	100	71	46	41	27	57.0%
Gemma 3 12B	100	79	68	36	0	56.6%
GPT-4o, Aug. 6th (temp=0)	100	90	62	16	14	56.3%
GPT-4.1	100	100	74	0	0	54.8%
GPT-4.1 Nano	100	96	44	28	0	53.5%
Llama 3.1 8B	100	98	31	28	0	51.4%
Claude Sonnet 4.6	90	89	41	34	0	50.9%
WizardLM 2 8x22b	90	85	80	0	0	50.9%
GPT-5 Nano	100	94	55	0	0	49.8%
GPT-4.1 Mini	100	100	34	0	0	46.7%
Claude 3.5 Sonnet	100	67	64	0	0	46.1%
Llama 3.1 Nemotron 70B	100	92	18	18	0	45.7%
Stealth: Aurora Alpha	100	78	41	0	0	43.8%
GPT-4o, Aug. 6th (temp=1)	96	71	21	14	0	40.1%
Claude 3.5 Haiku	100	100	0	0	0	40.0%
Claude Haiku 4.5	100	29	18	15	0	32.5%
Llama 3.1 70B	100	22	11	8	0	28.3%
Gemini 2.5 Flash	62	54	15	0	0	26.2%
GPT-5 Mini	44	27	25	10	5	22.1%
Gemini 2.5 Flash Lite	88	8	0	0	0	19.2%

▼

Mystery: examining a crime scene

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	97	99.4%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	93	98.6%
Grok 4	100	100	100	100	82	96.5%
Gemini 3 Pro (Preview)	100	100	100	94	88	96.4%
Mistral NeMO	100	100	100	100	81	96.1%
Qwen 3.5 397B A17B	100	100	94	93	92	95.9%
GPT-5 Mini	100	100	100	98	75	94.5%
Ministral 3 3B	100	100	100	100	72	94.4%
Gemini 3 Flash (Preview)	100	100	94	92	86	94.3%
Gemma 3 27B	100	100	100	100	67	93.3%
Claude Sonnet 4.6	100	100	100	100	62	92.5%
Arcee AI: Trinity Mini	100	100	100	100	61	92.1%
Cohere Command R+ (Aug. 2024)	100	100	100	100	57	91.5%
Claude 3.5 Sonnet	100	100	100	90	67	91.4%
Claude Sonnet 4	100	100	99	98	59	91.2%
GPT-4.1 Mini	100	100	100	100	52	90.4%
ByteDance Seed 1.6 Flash	100	100	100	95	55	90.0%
Claude Opus 4.6	100	100	94	78	73	89.0%
Mistral Large	100	100	100	92	48	88.0%
Z.AI GLM 4.7 Flash	100	100	100	100	40	87.9%
Claude 3 Haiku	100	100	100	97	41	87.6%
DeepSeek V3 (2024-12-26)	100	100	100	90	46	87.1%
GPT-5.2	100	100	100	82	52	87.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	34	86.7%
Llama 3.1 Nemotron 70B	100	100	100	69	64	86.6%
Gemma 3 4B	100	100	100	100	27	85.4%
DeepSeek V3.2	100	100	79	78	62	83.7%
MoonshotAI: Kimi K2.5	100	100	89	86	41	83.2%
Mistral Medium 3.1	100	100	100	68	46	82.8%
Gemini 2.5 Flash	100	100	93	84	36	82.8%
DeepSeek-V2 Chat	100	100	100	100	0	80.0%
Gemma 3 12B	100	100	100	100	0	80.0%
Llama 3.1 70B	100	100	100	100	0	80.0%
Mistral Small Creative	100	100	100	61	39	79.9%
GPT-4.1 Nano	100	100	100	59	39	79.6%
DeepSeek V3 (2025-03-24)	100	100	82	61	54	79.4%
Z.AI GLM 4.6	100	100	100	96	0	79.1%
WizardLM 2 8x22b	100	100	100	88	5	78.4%
Mistral Large 3	100	100	100	52	39	78.2%
GPT-4o, Aug. 6th (temp=1)	100	100	100	48	41	77.9%
Z.AI GLM 5	100	94	76	76	41	77.7%
DeepSeek V3.1	100	100	99	89	0	77.6%
Claude Sonnet 4.5	100	100	69	65	52	77.3%
Hermes 3 70B	100	100	100	57	11	73.7%
Claude 3.7 Sonnet	100	100	100	54	11	73.0%
Rocinante 12B	100	100	100	64	0	72.8%
Qwen 3.5 Plus (2026-02-15)	100	81	75	53	46	70.9%
Claude Haiku 4.5	100	100	100	50	0	70.0%
Ministral 3 14B	100	100	89	34	25	69.5%
Mistral Large 2	100	100	100	46	0	69.2%
GPT-4o Mini (temp=0)	100	100	82	52	6	68.0%
Gemini 2.5 Flash Lite	100	100	100	38	0	67.6%
Gemini 2.5 Pro	100	100	73	41	23	67.6%
Grok 4 Fast	100	100	100	35	0	66.9%
Qwen 2.5 72B	100	93	76	36	28	66.8%
Z.AI GLM 4.7	100	82	57	35	33	61.5%
Claude Opus 4.5	100	68	60	56	18	60.6%
Claude 3.5 Haiku	100	100	100	0	0	60.0%
Ministral 3B	100	100	67	16	3	57.1%
Z.AI GLM 4.5	100	96	89	0	0	56.9%
Arcee AI: Trinity Large (Preview)	100	97	74	0	0	54.2%
Ministral 8B	100	100	64	0	0	52.8%
Writer: Palmyra X5	100	100	31	28	0	51.8%
Llama 3.1 8B	100	100	54	0	0	50.8%
Minimax M2.5	100	52	50	39	0	48.2%
Stealth: Aurora Alpha	100	84	44	10	0	47.5%
Claude Opus 4	59	40	39	36	5	35.8%
GPT-5 Nano	48	0	0	0	0	9.5%

▼

Romance: separated couple reunites

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	97	99.4%
o4 Mini High	100	100	100	100	96	99.2%
DeepSeek V3 (2025-03-24)	100	100	100	99	97	99.2%
Mistral NeMO	100	100	100	100	92	98.4%
Mistral Small 3.2 24B	100	100	100	100	90	98.0%
Ministral 3B	100	100	100	99	82	96.2%
Grok 4 Fast	100	100	100	100	80	95.9%
Hermes 3 70B	100	100	100	100	74	94.8%
Claude Opus 4	100	100	100	96	76	94.4%
Writer: Palmyra X5	100	100	100	100	72	94.4%
DeepSeek V3.2	100	100	100	100	68	93.6%
ByteDance Seed 1.6 Flash	100	100	100	99	67	93.2%
Gemma 3 4B	100	100	100	100	65	93.1%
Mistral Large 2	100	100	100	93	72	93.0%
MoonshotAI: Kimi K2.5	100	100	100	100	64	92.8%
Mistral Large 3	100	100	100	100	64	92.8%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	61	92.1%
GPT-5.1	100	100	100	100	54	90.8%
Claude 3.5 Sonnet	100	100	100	100	50	90.0%
DeepSeek V3.1	100	94	90	90	68	88.4%
Ministral 3 8B	100	100	100	94	46	88.1%
Mistral Small Creative	100	100	100	69	62	86.3%
Gemma 3 12B	100	100	100	100	29	85.8%
Gemini 2.5 Pro	100	100	100	100	23	84.6%
GPT-4o, May 13th (temp=0)	100	100	100	65	57	84.5%
Claude Sonnet 4.5	100	100	89	84	48	84.2%
DeepSeek V3 (2024-12-26)	100	100	96	90	25	82.2%
Llama 3.1 Nemotron 70B	100	100	82	79	44	80.9%
Gemini 2.5 Flash	100	100	100	76	27	80.6%
Ministral 3 14B	100	100	100	98	3	80.2%
ByteDance Seed 1.6	100	100	100	100	0	80.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	0	80.0%
Claude 3.5 Haiku	100	100	100	100	0	80.0%
Cohere Command R+ (Aug. 2024)	100	100	100	100	0	80.0%
Z.AI GLM 5	100	100	87	73	38	79.6%
Claude 3.7 Sonnet	100	100	100	78	16	78.7%
WizardLM 2 8x22b	100	100	100	76	13	77.9%
Ministral 3 3B	100	100	100	46	29	75.0%
Rocinante 12B	100	100	75	54	33	72.3%
GPT-5.2	100	100	87	73	0	72.0%
Stealth: Aurora Alpha	100	98	87	42	32	71.9%
Z.AI GLM 4.7	100	100	82	43	31	71.2%
Gemini 3 Flash (Preview)	100	88	75	56	34	70.5%
Gemma 3 27B	100	100	72	54	21	69.3%
GPT-4o, Aug. 6th (temp=1)	100	100	100	46	0	69.2%
GPT-4.1 Nano	100	100	82	34	25	68.2%
Qwen 2.5 72B	100	100	96	39	0	66.9%
Z.AI GLM 4.7 Flash	100	76	72	40	40	65.4%
Claude Sonnet 4.6	100	100	88	15	0	60.5%
Arcee AI: Trinity Mini	100	100	54	39	6	59.7%
Claude 3 Haiku	100	96	54	48	0	59.5%
Minimax M2.5	100	89	71	18	8	57.3%
GPT-5	83	76	62	41	14	55.3%
Z.AI GLM 4.6	100	100	47	9	0	51.2%
Gemini 2.5 Flash Lite	100	95	44	5	0	48.9%
GPT-4o, Aug. 6th (temp=0)	100	59	57	18	0	47.0%
Claude Opus 4.6	100	75	40	15	0	46.0%
Claude Opus 4.5	100	82	25	5	0	42.5%
Llama 3.1 8B	100	100	11	0	0	42.2%
Claude Haiku 4.5	100	59	36	0	0	39.1%
GPT-5 Mini	100	54	30	0	0	36.7%
GPT-4.1 Mini	100	28	28	22	0	35.6%
Llama 3.1 70B	100	64	3	0	0	33.3%
GPT-5 Nano	9	0	0	0	0	1.8%

▼

Thriller: chase through city streets

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Arcee AI: Trinity Mini	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	98	99.6%
Mistral Large	100	100	100	100	96	99.1%
Z.AI GLM 4.7 Flash	100	100	100	96	91	97.4%
Rocinante 12B	100	100	100	100	79	95.7%
Ministral 3B	100	100	100	100	76	95.3%
GPT-5.2	100	100	100	99	75	94.8%
GPT-4o Mini (temp=1)	100	100	100	100	71	94.1%
MoonshotAI: Kimi K2.5	100	100	100	94	75	94.0%
WizardLM 2 8x22b	100	100	100	88	75	92.7%
Gemma 3 27B	100	100	100	82	81	92.6%
Writer: Palmyra X5	100	100	100	100	57	91.5%
Mistral Large 2	100	100	100	96	61	91.3%
ByteDance Seed 1.6	100	100	100	100	50	90.0%
Mistral NeMO	100	100	100	100	49	89.7%
Cohere Command R+ (Aug. 2024)	100	100	100	93	56	89.6%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	46	89.2%
Claude 3 Haiku	100	100	100	100	44	88.7%
ByteDance Seed 1.6 Flash	100	100	100	100	41	88.3%
GPT-4o Mini (temp=0)	100	100	100	72	69	88.2%
GPT-4o, May 13th (temp=1)	100	88	88	88	72	87.3%
Gemini 2.5 Pro	100	100	100	90	46	87.3%
Ministral 3 3B	100	100	92	89	54	86.9%
Gemma 3 12B	100	100	97	88	44	85.9%
GPT-4.1 Nano	100	100	100	100	29	85.8%
Ministral 8B	100	100	100	81	44	85.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	62	61	84.6%
DeepSeek V3.2	100	100	100	63	59	84.3%
DeepSeek-V2 Chat	100	100	100	62	57	83.8%
Z.AI GLM 5	100	100	94	66	59	83.7%
Claude Haiku 4.5	100	88	86	73	61	81.4%
Z.AI GLM 4.7	100	100	84	65	57	81.3%
Ministral 3 8B	100	100	100	100	0	80.0%
Claude Opus 4	100	100	100	59	40	79.8%
Claude Opus 4.6	100	100	85	65	47	79.3%
Llama 3.1 70B	100	94	93	82	15	77.0%
Qwen 3.5 Plus (2026-02-15)	100	100	93	50	41	76.8%
Z.AI GLM 4.5	100	100	100	59	23	76.4%
GPT-5	100	100	100	50	30	76.0%
DeepSeek V3 (2025-03-24)	100	100	86	64	28	75.5%
Gemini 3 Flash (Preview)	100	100	65	59	48	74.3%
GPT-5 Mini	100	100	95	55	22	74.3%
Gemma 3 4B	100	100	90	49	31	73.9%
Ministral 3 14B	100	96	94	44	34	73.5%
Gemini 3 Pro (Preview)	100	100	84	73	7	72.9%
Mistral Small Creative	100	100	69	64	29	72.4%
Llama 3.1 8B	100	98	97	61	0	71.1%
DeepSeek V3.1	100	100	79	41	34	70.7%
Mistral Medium 3.1	100	100	100	46	0	69.2%
Minimax M2.5	100	100	100	44	0	68.9%
Claude 3.5 Sonnet	100	100	89	39	7	67.0%
Gemini 2.5 Flash Lite	100	100	61	43	28	66.3%
Claude Sonnet 4.6	100	100	64	50	0	62.8%
Grok 4	81	78	77	76	0	62.5%
Gemini 2.5 Flash	100	91	74	25	17	61.4%
Claude Sonnet 4.5	79	73	57	41	18	53.8%
Z.AI GLM 4.6	100	100	55	3	0	51.6%
Claude 3.7 Sonnet	100	74	43	33	0	50.0%
Stealth: Aurora Alpha	83	81	69	7	0	47.9%
Llama 3.1 Nemotron 70B	100	100	22	15	0	47.3%
Claude 3.5 Haiku	100	100	3	0	0	40.6%
Claude Opus 4.5	93	64	0	0	0	31.4%
Claude Sonnet 4	67	31	21	18	0	27.3%
Arcee AI: Trinity Large (Preview)	52	36	20	0	0	21.7%
GPT-5 Nano	52	28	0	0	0	16.1%

Narrator intent-glossing

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Detailed Writing Rules

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

genre

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets

Novelcrafter Default Prompt

Fantasy: entering an ancient ruin

Horror: alone in an eerie place at night

Literary fiction: old friends reunite

Mystery: examining a crime scene

Romance: separated couple reunites

Thriller: chase through city streets