Either/Or composite

Test: Novel outline

Avg. Score

62.0%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Gemini 2.5 Flash Lite	100.0%	$0.0003	1.0s	100%
2	Claude 3.5 Haiku	100.0%	$0.0025	4.9s	100%
3	Z.AI GLM 4.5	100.0%	$0.0009	8.1s	100%
4	ByteDance Seed 1.6 Flash	100.0%	$0.0010	15.0s	100%
5	Writer: Palmyra X5	100.0%	$0.0039	12.4s	100%
6	Qwen 3.5 Plus (2026-02-15)	100.0%	$0.0021	16.3s	100%
7	GPT-4o, May 13th (temp=0)	100.0%	$0.013	7.2s	100%
8	Gemini 2.5 Pro	100.0%	$0.013	9.0s	100%
9	Gemini 3 Pro (Preview)	100.0%	$0.015	8.7s	100%
10	Z.AI GLM 5	100.0%	$0.0032	21.5s	100%
11	MoonshotAI: Kimi K2.5	100.0%	$0.0033	23.3s	100%
12	Gemini 3.1 Pro (Preview)	100.0%	$0.016	13.7s	100%
13	Llama 3.1 Nemotron 70B	95.0%	$0.0006	14.0s	70%
14	DeepSeek-V2 Chat	95.0%	$0.0003	17.9s	70%
15	Z.AI GLM 4.6	100.0%	$0.0023	34.1s	100%
16	Mistral Large	85.0%	$0.0037	2.0s	54%
17	GPT-5 Mini	90.0%	$0.0016	11.1s	60%
18	Mistral Large 3	90.0%	$0.0012	4.8s	40%
19	Minimax M2.5	85.0%	$0.0010	16.4s	54%
20	GPT-4o, Aug. 6th (temp=0)	75.0%	$0.0037	1.1s	38%
21	GPT-4.1	85.0%	$0.0046	7.3s	36%
22	GPT-4o Mini (temp=0)	50.0%	$0.0002	1.5s	50%
23	Gemma 3 12B	75.0%	$0.0002	9.5s	38%
24	GPT-5.1	85.0%	$0.0056	9.0s	36%
25	Qwen 3.5 397B A17B	100.0%	$0.0077	48.3s	100%
26	GPT-4o, Aug. 6th (temp=1)	75.0%	$0.0059	3.9s	33%
27	GPT-4o Mini (temp=1)	60.0%	$0.0003	1.9s	30%
28	GPT-4.1 Nano	65.0%	$0.0001	4.1s	27%
29	Claude Sonnet 4.6	80.0%	$0.0087	3.6s	20%
30	GPT-5	85.0%	$0.0093	14.9s	36%
31	Ministral 3 8B	50.0%	$0.0003	2.3s	28%
32	GPT-4.1 Mini	65.0%	$0.0007	5.8s	16%
33	Gemini 2.5 Flash	55.0%	$0.0005	886ms	15%
34	Qwen 2.5 72B	70.0%	$0.0008	14.9s	25%
35	Claude Sonnet 4	70.0%	$0.0099	5.6s	25%
36	o4 Mini	70.0%	$0.0049	11.6s	25%
37	Claude Haiku 4.5	65.0%	$0.0032	3.1s	10%
38	o4 Mini High	70.0%	$0.0054	13.6s	25%
39	Ministral 3 3B	40.0%	$0.0002	1.5s	20%
40	Gemini 3 Flash (Preview)	40.0%	$0.0013	1.7s	20%
41	Mistral Large 2	50.0%	$0.0048	7.9s	28%
42	Mistral Small Creative	40.0%	$0.0003	3.6s	20%
43	Grok 4.1 Fast	35.0%	$0.0006	7.0s	27%
44	Cohere Command R+ (Aug. 2024)	40.0%	$0.0046	2.4s	20%
45	Claude Opus 4	95.0%	$0.047	11.4s	70%
46	Z.AI GLM 4.7	90.0%	$0.0029	51.1s	60%
47	Ministral 3B	40.0%	$0.0001	3.9s	13%
48	WizardLM 2 8x22b	55.0%	$0.0019	11.7s	15%
49	Ministral 3 14B	35.0%	$0.0004	5.0s	18%
50	GPT-5 Nano	70.0%	$0.0009	26.9s	26%
51	Arcee AI: Trinity Mini	30.0%	$0.0002	6.7s	26%
52	GPT-4o, May 13th (temp=1)	65.0%	$0.014	7.2s	16%
53	ByteDance Seed 1.6	75.0%	$0.0034	34.0s	38%
54	Llama 3.1 70B	65.0%	$0.0012	17.6s	10%
55	DeepSeek V3 (2024-12-26)	55.0%	$0.0008	10.1s	4%
56	Hermes 3 70B	55.0%	$0.0006	10.8s	4%
57	DeepSeek V3.1	55.0%	$0.0005	13.6s	8%
58	Claude Opus 4.5	55.0%	$0.016	4.9s	23%
59	Stealth: Aurora Alpha	50.0%	—	937ms	28%
60	Llama 3.1 8B	35.0%	$0.0002	2.6s	5%
61	Gemma 3 27B	40.0%	$0.0002	9.7s	13%
62	Mistral NeMO	35.0%	$0.0003	3.5s	5%
63	Claude 3 Haiku	35.0%	$0.0006	1.6s	0%
64	Claude Sonnet 4.5	50.0%	$0.0094	4.5s	5%
65	Rocinante 12B	35.0%	$0.0005	13.5s	18%
66	Arcee AI: Trinity Large (Preview)	25.0%	$0.0000	2.1s	0%
67	Z.AI GLM 4.7 Flash	70.0%	$0.0010	39.3s	25%
68	Grok 4 Fast	25.0%	$0.0005	3.8s	0%
69	DeepSeek V3.2	40.0%	$0.0005	13.5s	3%
70	Hermes 3 405B	25.0%	$0.0000	11.9s	13%
71	Gemma 3 4B	15.0%	$0.0001	3.5s	0%
72	DeepSeek V3 (2025-03-24)	25.0%	$0.0008	9.5s	0%
73	Claude Opus 4.6	40.0%	$0.012	5.5s	0%
74	Claude 3.7 Sonnet	20.0%	$0.0062	3.5s	0%
75	Mistral Small 3.2 24B	5.0%	$0.0002	3.0s	0%
76	Grok 4	60.0%	$0.017	20.2s	13%
77	Ministral 8B	15.0%	$0.0002	8.4s	0%
78	Mistral Medium 3.1	10.0%	$0.0013	5.3s	0%
79	GPT-5.2	20.0%	$0.0055	6.4s	0%
80	Claude 3.5 Sonnet	10.0%	$0.0062	5.2s	0%
62.00%

Individual Scenarios

pov-count

▼

Count point of views for Jack and Olivia

Model	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	Avg ▼
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100	100	100	100	100	100.0%
Claude 3.5 Haiku	100	100	100	100	100	100	100	100	100	100	100.0%
ByteDance Seed 1.6 Flash	100	100	100	100	100	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite	100	100	100	100	100	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100	100	100	100	50	95.0%
DeepSeek-V2 Chat	100	100	100	100	100	100	100	100	100	50	95.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100	100	100	100	50	95.0%
GPT-5 Mini	100	100	100	100	100	100	100	100	50	50	90.0%
Z.AI GLM 4.7	100	100	100	100	100	100	100	100	50	50	90.0%
Mistral Large 3	100	100	100	100	100	100	100	100	100	0	90.0%
GPT-5.1	100	100	100	100	100	100	100	100	50	0	85.0%
GPT-5	100	100	100	100	100	100	100	100	50	0	85.0%
Minimax M2.5	100	100	100	100	100	100	100	50	50	50	85.0%
GPT-4.1	100	100	100	100	100	100	100	100	50	0	85.0%
Mistral Large	100	100	100	100	100	100	100	50	50	50	85.0%
Claude Sonnet 4.6	100	100	100	100	100	100	100	100	0	0	80.0%
ByteDance Seed 1.6	100	100	100	100	100	50	50	50	50	50	75.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100	50	50	50	0	75.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	50	50	50	50	50	75.0%
Gemma 3 12B	100	100	100	100	100	50	50	50	50	50	75.0%
o4 Mini High	100	100	100	100	100	50	50	50	50	0	70.0%
o4 Mini	100	100	100	100	100	50	50	50	50	0	70.0%
Claude Sonnet 4	100	100	100	100	100	50	50	50	50	0	70.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	50	50	50	50	0	70.0%
GPT-5 Nano	100	100	100	100	50	50	50	50	50	50	70.0%
Qwen 2.5 72B	100	100	100	100	100	50	50	50	50	0	70.0%
Claude Haiku 4.5	100	100	100	100	100	100	50	0	0	0	65.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	50	50	50	0	0	65.0%
GPT-4.1 Mini	100	100	100	100	100	50	50	50	0	0	65.0%
Llama 3.1 70B	100	100	100	100	100	100	50	0	0	0	65.0%
GPT-4.1 Nano	100	100	100	50	50	50	50	50	50	50	65.0%
Grok 4	100	100	100	100	50	50	50	50	0	0	60.0%
GPT-4o Mini (temp=1)	100	100	50	50	50	50	50	50	50	50	60.0%
Claude Opus 4.5	100	100	50	50	50	50	50	50	50	0	55.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	50	0	0	0	0	55.0%
DeepSeek V3.1	100	100	100	100	50	50	50	0	0	0	55.0%
Gemini 2.5 Flash	100	100	100	50	50	50	50	50	0	0	55.0%
Hermes 3 70B	100	100	100	100	100	50	0	0	0	0	55.0%
WizardLM 2 8x22b	100	100	100	50	50	50	50	50	0	0	55.0%
Claude Sonnet 4.5	100	100	100	100	50	50	0	0	0	0	50.0%
Stealth: Aurora Alpha	100	50	50	50	50	50	50	50	50	0	50.0%
Mistral Large 2	100	50	50	50	50	50	50	50	50	0	50.0%
GPT-4o Mini (temp=0)	50	50	50	50	50	50	50	50	50	50	50.0%
Ministral 3 8B	100	50	50	50	50	50	50	50	50	0	50.0%
Claude Opus 4.6	100	100	100	100	0	0	0	0	0	0	40.0%
Gemini 3 Flash (Preview)	100	50	50	50	50	50	50	0	0	0	40.0%
DeepSeek V3.2	100	100	100	50	50	0	0	0	0	0	40.0%
Gemma 3 27B	100	100	50	50	50	50	0	0	0	0	40.0%
Mistral Small Creative	100	50	50	50	50	50	50	0	0	0	40.0%
Ministral 3 3B	100	50	50	50	50	50	50	0	0	0	40.0%
Cohere Command R+ (Aug. 2024)	100	50	50	50	50	50	50	0	0	0	40.0%
Ministral 3B	100	100	50	50	50	50	0	0	0	0	40.0%
Grok 4.1 Fast	50	50	50	50	50	50	50	0	0	0	35.0%
Ministral 3 14B	100	50	50	50	50	50	0	0	0	0	35.0%
Claude 3 Haiku	100	100	100	50	0	0	0	0	0	0	35.0%
Llama 3.1 8B	100	100	50	50	50	0	0	0	0	0	35.0%
Mistral NeMO	100	100	50	50	50	0	0	0	0	0	35.0%
Rocinante 12B	100	50	50	50	50	50	0	0	0	0	35.0%
Arcee AI: Trinity Mini	50	50	50	50	50	50	0	0	0	0	30.0%
DeepSeek V3 (2025-03-24)	100	50	50	50	0	0	0	0	0	0	25.0%
Grok 4 Fast	100	50	50	50	0	0	0	0	0	0	25.0%
Hermes 3 405B	50	50	50	50	50	0	0	0	0	0	25.0%
Arcee AI: Trinity Large (Preview)	100	50	50	50	0	0	0	0	0	0	25.0%
GPT-5.2	100	100	0	0	0	0	0	0	0	0	20.0%
Claude 3.7 Sonnet	100	50	50	0	0	0	0	0	0	0	20.0%
Gemma 3 4B	50	50	50	0	0	0	0	0	0	0	15.0%
Ministral 8B	100	50	0	0	0	0	0	0	0	0	15.0%
Claude 3.5 Sonnet	50	50	0	0	0	0	0	0	0	0	10.0%
Mistral Medium 3.1	50	50	0	0	0	0	0	0	0	0	10.0%
Mistral Small 3.2 24B	50	0	0	0	0	0	0	0	0	0	5.0%

Either/Or composite

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

pov-count

Count point of views for Jack and Olivia