5 codex entries

Codex Violation Detection

Detects factual inconsistencies between a story bible and prose passages. The model must output structured XML identifying each violation with paragraph number and substring.

Tooling Reasoning

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.1 Pro (Preview)	100%
Claude Sonnet 4.6 (Reasoning)	100%
GPT-5.4 (Reasoning)	100%
GPT-5 Mini	100%
GPT-5.1	100%
Claude Opus 4.6	100%
Qwen 3.5 122B	100%
Grok 4.20 (Beta, Reasoning)	100%
GPT-5.4 (Reasoning, Low)	100%
Z.AI GLM 5	100%
MoonshotAI: Kimi K2.5	100%
ByteDance Seed 1.6	100%
GPT-5.4 Mini (Reasoning)	100%
Gemini 3 Flash (Preview, Reasoning)	100%
o4 Mini High	100%
GPT-5.2	100%
Claude Opus 4.5	100%
Grok 4.1 Fast	100%
Aion 2.0	100%

	Score	Cost	Time
Gemini 3.1 Flash Lite (Preview)	97%	$0.0008	1.3s
Gemini 3 Flash (Preview)	100%	$0.0015	2.3s
Gemini 2.5 Flash Lite (Reasoning)	98%	$0.0008	5.2s
Grok 4 Fast	99%	$0.0008	6.9s
DeepSeek-V2 Chat	96%	$0.0007	6.2s
DeepSeek V3 (2024-12-26)	96%	$0.0007	7.3s
GPT-5.4 Nano (Reasoning)	89%	$0.0014	6.3s
Stealth: Healer Alpha	100%	$0.0000	12.7s
Qwen 3 32B	100%	$0.0005	13.1s
Grok 4.1 Fast	100%	$0.0009	9.2s
Claude 3.5 Haiku	97%	$0.0022	5.3s
Stealth: Aurora Alpha	94%	—	2.7s
Qwen 3.5 Plus (2026-02-15)	97%	$0.0014	12.5s
GPT-5.4 Mini (Reasoning, Low)	100%	$0.0032	3.8s
Llama 3.1 Nemotron 70B	96%	$0.0020	8.2s
GPT-4.1	100%	$0.0035	3.2s
Hermes 3 405B	100%	$0.0016	10.8s
DeepSeek V3 (2025-03-24)	99%	$0.0006	9.6s
Llama 3.1 70B	95%	$0.0008	10.8s
Z.AI GLM 4.6	97%	$0.0014	23.7s

	Score	Cost	Speed	Stability
Gemini 3 Flash (Preview)	100%	$0.0015	2.3s	100%
Grok 4.1 Fast	100%	$0.0009	9.2s	100%
Stealth: Healer Alpha	100%	$0.0000	12.7s	100%
GPT-5.4 Mini (Reasoning, Low)	100%	$0.0032	3.8s	100%
GPT-4.1	100%	$0.0035	3.2s	100%
Qwen 3 32B	100%	$0.0005	13.1s	100%
Hermes 3 405B	100%	$0.0016	10.8s	100%
Gemini 3.1 Flash Lite (Preview)	97%	$0.0008	1.3s	92%
Grok 4 Fast	99%	$0.0008	6.9s	95%
Gemini 2.5 Flash (Reasoning)	100%	$0.0037	5.9s	100%
Gemini 2.5 Flash Lite (Reasoning)	98%	$0.0008	5.2s	94%
GPT-5.4	100%	$0.0051	3.7s	100%
DeepSeek V3 (2025-03-24)	99%	$0.0006	9.6s	95%
Z.AI GLM 4.5	99%	$0.0013	11.9s	95%
GPT-5.4 Mini (Reasoning)	100%	$0.0055	7.3s	100%
Z.AI GLM 5 Turbo	99%	$0.0031	7.6s	95%
Gemini 3 Flash (Preview, Reasoning)	100%	$0.0055	8.9s	100%
Inception Mercury 2	95%	$0.0011	1.8s	88%
DeepSeek-V2 Chat	96%	$0.0007	6.2s	89%
Claude Sonnet 4	100%	$0.0081	4.2s	100%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	Total
95	Claude Opus 4.6 (Reasoning)	$0.028	18.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
88	Gemini 3.1 Pro (Preview)	$0.024	20.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
65	Claude Sonnet 4.6 (Reasoning)	$0.018	12.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
53	GPT-5.4 (Reasoning)	$0.013	13.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
38	GPT-5 Mini	$0.0051	25.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
63	GPT-5.1	$0.014	21.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
48	Claude Opus 4.6	$0.014	4.7s	100%	100	100	100	100	100	100	100	100	100	100	100%
82	Qwen 3.5 122B	$0.014	42.6s	100%	100	100	100	100	100	100	100	100	100	100	100%
56	Grok 4.20 (Beta, Reasoning)	$0.015	10.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
25	GPT-5.4 (Reasoning, Low)	$0.0085	5.9s	100%	100	100	100	100	100	100	100	100	100	100	100%
28	Z.AI GLM 5	$0.0047	18.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
66	MoonshotAI: Kimi K2.5	$0.0078	45.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
31	ByteDance Seed 1.6	$0.0026	25.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
15	GPT-5.4 Mini (Reasoning)	$0.0055	7.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
17	Gemini 3 Flash (Preview, Reasoning)	$0.0055	8.9s	100%	100	100	100	100	100	100	100	100	100	100	100%
74	o4 Mini High	$0.016	29.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
33	GPT-5.2	$0.0090	9.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
47	Claude Opus 4.5	$0.014	4.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
2	Grok 4.1 Fast	$0.0009	9.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
54	Aion 2.0	$0.0041	42.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
80	Gemini 3 Pro (Preview)	$0.023	15.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
20	Claude Sonnet 4	$0.0081	4.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
55	Z.AI GLM 4.7	$0.0053	39.6s	100%	100	100	100	100	100	100	100	100	100	100	100%
5	GPT-4.1	$0.0035	3.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
62	Gemini 2.5 Pro	$0.018	11.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
42	o4 Mini	$0.010	13.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
59	Qwen 3.5 35B	$0.0094	32.4s	100%	100	100	100	100	100	100	100	100	100	100	100%
72	ByteDance Seed 2.0 Mini	$0.0013	1.2m	100%	100	100	100	100	100	100	100	100	100	100	100%
10	Gemini 2.5 Flash (Reasoning)	$0.0037	5.9s	100%	100	100	100	100	100	100	100	100	100	100	100%
35	Qwen 3.5 Flash	$0.0022	34.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
3	Stealth: Healer Alpha	$0.0000	12.7s	100%	100	100	100	100	100	100	100	100	100	100	100%
4	GPT-5.4 Mini (Reasoning, Low)	$0.0032	3.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
1	Gemini 3 Flash (Preview)	$0.0015	2.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
41	ByteDance Seed 2.0 Lite	$0.0032	34.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
37	Nemotron 3 Super	$0.0000	41.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
12	GPT-5.4	$0.0051	3.7s	100%	100	100	100	100	100	100	100	100	100	100	100%
57	Claude 3.5 Sonnet	$0.017	5.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
23	Claude 3.7 Sonnet	$0.0087	4.6s	100%	100	100	100	100	100	100	100	100	100	100	100%
7	Hermes 3 405B	$0.0016	10.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
6	Qwen 3 32B	$0.0005	13.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
16	Z.AI GLM 5 Turbo	$0.0031	7.6s	95%	100	100	100	100	100	100	100	100	100	91	99%
113	Qwen 3.5 397B A17B	$0.017	2.0m	95%	100	100	100	100	100	100	100	100	100	91	99%
103	Qwen 3.5 27B	$0.015	1.4m	95%	100	100	100	100	100	100	100	100	100	91	99%
14	Z.AI GLM 4.5	$0.0013	11.9s	95%	100	100	100	100	100	100	100	100	100	91	99%
9	Grok 4 Fast	$0.0008	6.9s	95%	100	100	100	100	100	100	100	100	100	91	99%
13	DeepSeek V3 (2025-03-24)	$0.0006	9.6s	95%	100	100	100	100	100	100	100	100	93	93	99%
90	Qwen 3.5 9B	$0.0011	1.4m	94%	100	100	100	100	100	100	100	100	93	91	98%
39	Claude Sonnet 4.5	$0.0081	4.2s	90%	100	100	100	100	100	100	100	100	100	83	98%
11	Gemini 2.5 Flash Lite (Reasoning)	$0.0008	5.2s	94%	100	100	100	100	100	100	100	93	93	93	98%
43	Z.AI GLM 4.6	$0.0014	23.7s	89%	100	100	100	100	100	100	100	100	91	83	97%
104	Grok 4	$0.029	41.0s	92%	100	100	100	100	100	100	100	91	91	91	97%
8	Gemini 3.1 Flash Lite (Preview)	$0.0008	1.3s	92%	100	100	100	100	100	100	100	91	91	91	97%
27	Claude 3.5 Haiku	$0.0022	5.3s	87%	100	100	100	100	100	100	100	100	83	83	97%
60	Nemotron 3 Nano	$0.0009	46.3s	91%	100	100	100	100	100	100	93	93	93	87	97%
46	Qwen 3.5 Plus (2026-02-15)	$0.0014	12.5s	79%	100	100	100	100	100	100	100	100	100	66	97%
116	GPT-5	$0.040	1.1m	91%	100	100	100	100	100	100	91	91	91	91	96%
40	Stealth: Hunter Alpha	$0.0000	28.6s	91%	100	100	100	100	100	100	91	91	91	91	96%
19	DeepSeek-V2 Chat	$0.0007	6.2s	89%	100	100	100	100	100	100	93	93	91	83	96%
44	Llama 3.1 Nemotron 70B	$0.0020	8.2s	79%	100	100	100	100	100	100	100	100	91	66	96%
30	DeepSeek V3 (2024-12-26)	$0.0007	7.3s	85%	100	100	100	100	100	100	93	93	93	76	96%
21	DeepSeek V3.2	$0.0004	7.6s	87%	100	100	100	100	100	91	91	91	91	91	96%
18	Inception Mercury 2	$0.0011	1.8s	88%	100	100	100	93	93	93	93	93	93	93	95%
32	Llama 3.1 70B	$0.0008	10.8s	87%	100	100	100	100	100	100	93	90	83	83	95%
50	Stealth: Aurora Alpha	—	2.7s	77%	100	100	100	100	100	93	93	93	93	66	94%
45	Mistral Large	$0.0046	4.7s	84%	100	100	100	91	91	91	91	91	91	91	94%
49	MiniMax M2.7	$0.0011	16.4s	83%	100	100	100	93	93	93	93	93	83	83	93%
34	Mistral Small 4 (Reasoning)	$0.0010	9.0s	85%	100	100	93	93	93	93	91	91	91	83	93%
26	ByteDance Seed 1.6 Flash	$0.0004	5.7s	86%	100	93	93	93	93	93	93	93	93	83	93%
86	Z.AI GLM 4.7 Flash	$0.0010	52.6s	80%	100	100	100	100	100	93	93	83	83	75	93%
24	Inception Mercury	$0.0002	5.9s	87%	100	93	93	93	93	93	93	93	87	87	93%
29	Gemini 2.5 Flash	$0.0009	1.2s	84%	100	93	93	93	93	93	93	93	83	83	92%
110	Claude Opus 4	$0.040	6.7s	76%	100	100	100	100	100	83	83	83	83	83	92%
22	Mistral Large 3	$0.0012	5.7s	91%	91	91	91	91	91	91	91	91	91	91	91%
36	Mistral Large 2	$0.0047	5.1s	91%	91	91	91	91	91	91	91	91	91	91	91%
76	GPT-4o, May 13th (temp=0)	$0.011	2.0s	72%	100	100	100	100	91	91	83	76	76	76	89%
89	GPT-5.4 Nano (Reasoning)	$0.0014	6.3s	41%	100	100	100	100	100	100	100	97	93	0	89%
93	DeepSeek V3.1	$0.0005	14.7s	43%	100	100	100	100	100	100	91	84	83	3	86%
52	Qwen 2.5 72B	$0.0002	5.8s	77%	93	93	93	93	93	93	75	75	75	75	86%
71	MiniMax M2.5	$0.0012	16.1s	68%	100	100	93	91	83	83	83	75	75	75	86%
58	Mistral Medium 3.1	$0.0011	3.8s	73%	100	83	83	83	83	83	83	83	76	76	83%
64	Arcee AI: Trinity Mini	$0.0002	4.6s	66%	100	93	93	93	87	87	75	75	66	66	83%
67	Claude Sonnet 4.6	$0.0087	4.1s	83%	83	83	83	83	83	83	83	83	83	83	83%
51	Gemma 3 27B	$0.0002	8.7s	83%	83	83	83	83	83	83	83	83	83	83	83%
61	GPT-4.1 Mini	$0.0010	3.2s	69%	93	93	83	83	83	83	83	83	83	60	83%
114	GPT-5 Nano	$0.0033	1.3m	41%	100	100	100	93	93	93	93	75	75	3	83%
75	GPT-4o Mini (temp=1)	$0.0003	5.4s	59%	93	93	93	93	75	75	75	75	67	67	81%
92	GPT-4o, May 13th (temp=1)	$0.011	2.7s	66%	91	91	83	83	76	76	76	76	76	76	80%
81	GPT-4o, Aug. 6th (temp=0)	$0.0054	2.2s	65%	91	91	91	76	76	76	76	76	76	76	80%
79	GPT-4o, Aug. 6th (temp=1)	$0.0052	2.4s	66%	91	91	83	76	76	76	76	76	76	76	79%
70	GPT-4o Mini (temp=0)	$0.0003	5.0s	64%	93	93	75	75	75	75	75	75	75	75	79%
68	Grok 4.20 (Beta)	$0.0024	1.3s	69%	84	83	83	76	76	76	76	76	76	69	77%
84	Writer: Palmyra X5	$0.0028	7.2s	64%	83	83	83	83	83	83	76	76	76	44	77%
69	Gemma 3 12B	$0.0001	5.5s	67%	93	75	75	75	75	75	75	75	75	75	77%
85	Qwen3 235B A22B Instruct 2507	$0.0002	10.9s	62%	83	83	83	83	83	76	76	76	76	44	76%
78	Claude Haiku 4.5	$0.0029	3.1s	65%	83	83	83	83	76	76	75	67	67	67	76%
77	Arcee AI: Trinity Large (Preview)	$0.0000	8.1s	64%	83	83	83	83	76	76	69	69	67	63	75%
87	GPT-5.4 Mini	$0.0023	1.9s	59%	91	83	83	76	76	76	76	67	67	50	74%
73	Mistral Small Creative	$0.0002	2.1s	67%	76	76	76	76	76	76	69	69	63	60	71%
100	Cohere Command R+ (Aug. 2024)	$0.0057	5.8s	49%	93	87	87	76	75	67	57	57	57	44	70%
83	Gemini 2.5 Flash Lite	$0.0002	947ms	62%	75	75	67	67	67	67	67	67	67	67	68%
94	Mistral NeMO	$0.0002	2.0s	51%	83	75	75	75	67	67	60	60	50	44	66%
99	Llama 3.1 8B	$0.0001	4.6s	45%	83	83	78	78	63	60	60	57	54	39	65%
96	Ministral 3 3B	$0.0002	1.6s	49%	83	83	76	69	67	67	63	60	44	42	65%
91	Mistral Small 3.2 24B	$0.0002	4.5s	58%	76	69	69	67	67	67	60	60	60	50	64%
97	Claude 3 Haiku	$0.0007	3.3s	52%	76	76	67	67	67	60	57	57	50	50	62%
101	Ministral 3B	$0.0001	1.2s	40%	83	78	76	67	57	54	50	50	44	42	60%
98	Ministral 3 14B	$0.0004	4.1s	52%	69	69	63	63	63	54	54	54	54	53	60%
102	Mistral Small 4	$0.0003	2.2s	37%	75	75	67	67	67	60	60	54	44	0	57%
108	Hermes 3 70B	$0.0005	13.2s	34%	76	75	75	66	60	57	57	50	44	0	56%
111	GPT-5.4 Nano (Reasoning, Low)	$0.0009	3.0s	7%	100	100	100	93	60	54	0	0	0	0	51%
105	Ministral 8B	$0.0002	3.6s	37%	69	49	49	44	44	43	43	43	43	42	47%
109	GPT-5.4 Nano	$0.0005	1.7s	23%	76	76	75	69	67	60	43	0	0	0	47%
106	Gemma 3 4B	$0.0001	2.6s	36%	57	57	57	44	42	42	42	42	42	42	46%
107	Ministral 3 8B	$0.0003	3.8s	40%	49	44	44	44	43	42	40	40	40	39	43%
115	Rocinante 12B	$0.0004	5.1s	21%	67	57	53	40	38	37	33	3	3	3	34%
112	GPT-4.1 Nano	$0.0001	951ms	20%	57	57	57	57	44	33	17	0	0	0	32%
118	WizardLM 2 8x22b	$0.0016	14.5s	3%	3	3	3	3	3	3	0	0	0	0	2%
117	LFM2 24B	$0.0000	2.3s	0%	0	0	0	0	0	0	0	0	0	0	0%
86.36%

Median	Evaluator	Top 3	Flop 3
92.0%	Accuracy (recall)	100GPT-5.4 100Claude Opus 4.5 100Gemini 2.5 Flash (Reasoning)	0WizardLM 2 8x22b 0LFM2 24B 1Ministral 3 8B
96.0%	Precision	100GPT-5.4 Mini (Reasoning, Low) 100GPT-5.1 100Stealth: Healer Alpha	0WizardLM 2 8x22b 0LFM2 24B 21Rocinante 12B
100.0%	Structural validity	100Claude Opus 4.6 100DeepSeek V3 (2024-12-26) 100Inception Mercury	0LFM2 24B 6WizardLM 2 8x22b 45GPT-4.1 Nano

Codex Violation Detection

5 codex entries

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)