40 codex entries

Codex Violation Detection

Detects factual inconsistencies between a story bible and prose passages. The model must output structured XML identifying each violation with paragraph number and substring.

Tooling Reasoning

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Claude Opus 4.6 (Reasoning)	100%
Qwen3.6 Max Preview	100%
Gemini 3.1 Pro (Preview)	100%
Claude Sonnet 4.6 (Reasoning)	100%
Claude Opus 4.7 (Reasoning)	100%
GPT-5.5 (Reasoning)	100%
Claude Opus 4.6	100%
Qwen 3.5 Plus (2026-04-20)	100%
Grok 4.20 (Beta, Reasoning)	100%
GPT-5.4 (Reasoning, Low)	100%
Grok 4.20 (Reasoning)	100%
Grok 4.1 Fast	100%
GPT-5.5	100%
Qwen 3.6 35B	100%
Grok 4	100%
Gemma 4 31B	100%
Grok 4 Fast	100%
Z.AI GLM 5 Turbo	100%
Qwen 3.5 35B	100%
GPT-5.5 (Reasoning, Low)	100%

	Score	Cost	Time
Stealth: Healer Alpha	98%	$0.0000	16.1s
Grok 4.1 Fast	100%	$0.0016	10.1s
Gemma 4 31B	100%	$0.0012	29.3s
Grok 4 Fast	100%	$0.0021	12.1s
DeepSeek V4 Flash (Reasoning)	99%	$0.0010	36.2s
Stealth: Hunter Alpha	97%	$0.0000	32.5s
Qwen 3.5 Flash	98%	$0.0036	47.1s
Z.AI GLM 5 Turbo	100%	$0.0057	13.3s
Gemini 2.5 Flash Lite (Reasoning)	97%	$0.0016	11.6s
Xiaomi MIMO v2.5	97%	$0.0045	16.2s
DeepSeek V4 Pro (Reasoning)	89%	$0.0098	1.8m
Xiaomi MIMO v2.5 Pro	99%	$0.0082	28.3s
Gemini 2.5 Flash	96%	$0.0027	3.3s
Z.AI GLM 4.7	98%	$0.0070	28.9s
Gemma 4 26B (Reasoning)	99%	$0.0024	1.5m
Nemotron 3 Super	97%	$0.0000	2.8m
Gemma 4 31B (Reasoning)	99%	$0.0019	2.6m
Inception Mercury 2	94%	$0.0029	4.4s
Gemini 2.5 Flash (Reasoning)	98%	$0.0072	10.7s
GPT-5 Mini	99%	$0.0075	1.3m

	Score	Cost	Speed	Stability
Grok 4.1 Fast	100%	$0.0016	10.1s	100%
Grok 4 Fast	100%	$0.0021	12.1s	100%
Gemma 4 31B	100%	$0.0012	29.3s	100%
Z.AI GLM 5 Turbo	100%	$0.0057	13.3s	98%
Stealth: Healer Alpha	98%	$0.0000	16.1s	95%
GPT-5.4 (Reasoning, Low)	100%	$0.017	11.0s	100%
GPT-5.4	99%	$0.011	7.9s	96%
DeepSeek V4 Flash (Reasoning)	99%	$0.0010	36.2s	95%
Gemini 2.5 Flash (Reasoning)	98%	$0.0072	10.7s	94%
GPT-5.5	100%	$0.022	5.8s	100%
Grok 4.20 (Reasoning)	100%	$0.015	33.6s	100%
Gemini 2.5 Flash Lite (Reasoning)	97%	$0.0016	11.6s	91%
Xiaomi MIMO v2.5 Pro	99%	$0.0082	28.3s	95%
Xiaomi MIMO v2.5	97%	$0.0045	16.2s	93%
GPT-5.4 Mini (Reasoning)	99%	$0.013	18.6s	97%
Qwen 3.6 35B	100%	$0.013	48.4s	100%
Grok 4.20 (Beta, Reasoning)	100%	$0.022	13.0s	100%
Qwen 3.6 Flash	99%	$0.010	26.2s	95%
Gemini 2.5 Flash	96%	$0.0027	3.3s	90%
Stealth: Hunter Alpha	97%	$0.0000	32.5s	91%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	Total
94	Claude Opus 4.6 (Reasoning)	$0.077	25.5s	100%	100	100	100	100	100	100	100	100	100	100	100%
90	Qwen3.6 Max Preview	$0.046	2.0m	100%	100	100	100	100	100	100	100	100	100	100	100%
68	Gemini 3.1 Pro (Preview)	$0.051	34.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
74	Claude Sonnet 4.6 (Reasoning)	$0.058	28.7s	100%	100	100	100	100	100	100	100	100	100	100	100%
92	Claude Opus 4.7 (Reasoning)	$0.079	9.2s	100%	100	100	100	100	100	100	100	100	100	100	100%
78	GPT-5.5 (Reasoning)	$0.062	24.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
52	Claude Opus 4.6	$0.050	8.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
28	Qwen 3.5 Plus (2026-04-20)	$0.014	1.2m	100%	100	100	100	100	100	100	100	100	100	100	100%
17	Grok 4.20 (Beta, Reasoning)	$0.022	13.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
6	GPT-5.4 (Reasoning, Low)	$0.017	11.0s	100%	100	100	100	100	100	100	100	100	100	100	100%
11	Grok 4.20 (Reasoning)	$0.015	33.6s	100%	100	100	100	100	100	100	100	100	100	100	100%
1	Grok 4.1 Fast	$0.0016	10.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
10	GPT-5.5	$0.022	5.8s	100%	100	100	100	100	100	100	100	100	100	100	100%
16	Qwen 3.6 35B	$0.013	48.4s	100%	100	100	100	100	100	100	100	100	100	100	100%
66	Grok 4	$0.045	50.7s	100%	100	100	100	100	100	100	100	100	100	100	100%
3	Gemma 4 31B	$0.0012	29.3s	100%	100	100	100	100	100	100	100	100	100	100	100%
2	Grok 4 Fast	$0.0021	12.1s	100%	100	100	100	100	100	100	100	100	100	100	100%
4	Z.AI GLM 5 Turbo	$0.0057	13.3s	98%	100	100	100	100	100	100	100	100	100	97	100%
22	Qwen 3.5 35B	$0.015	41.5s	98%	100	100	100	100	100	100	100	100	100	97	100%
44	GPT-5.5 (Reasoning, Low)	$0.034	12.1s	97%	100	100	100	100	100	100	100	100	100	95	100%
37	Qwen 3.5 27B	$0.015	1.1m	97%	100	100	100	100	100	100	100	100	100	95	100%
33	Z.AI GLM 5.1	$0.013	1.1m	97%	100	100	100	100	100	100	100	100	97	97	99%
54	Grok 4.3 (Reasoning)	$0.021	1.4m	95%	100	100	100	100	100	100	100	100	100	92	99%
55	Gemma 4 31B (Reasoning)	$0.0019	2.6m	95%	100	100	100	100	100	100	100	100	100	92	99%
18	Qwen 3.6 Flash	$0.010	26.2s	95%	100	100	100	100	100	100	100	100	100	92	99%
40	Gemini 2.5 Pro	$0.027	16.8s	95%	100	100	100	100	100	100	100	100	100	92	99%
41	Claude Sonnet 4.5	$0.030	7.3s	95%	100	100	100	100	100	100	100	100	100	92	99%
13	Xiaomi MIMO v2.5 Pro	$0.0082	28.3s	95%	100	100	100	100	100	100	100	100	100	92	99%
30	GPT-5 Mini	$0.0075	1.3m	96%	100	100	100	100	100	100	100	100	95	95	99%
7	GPT-5.4	$0.011	7.9s	96%	100	100	100	100	100	100	100	100	95	95	99%
29	Gemma 4 26B (Reasoning)	$0.0024	1.5m	97%	100	100	100	100	100	100	100	97	97	97	99%
63	GPT-5.4 (Reasoning)	$0.039	39.6s	96%	100	100	100	100	100	100	100	97	97	95	99%
8	DeepSeek V4 Flash (Reasoning)	$0.0010	36.2s	95%	100	100	100	100	100	100	100	100	97	92	99%
35	MoonshotAI: Kimi K2.5	$0.012	54.2s	95%	100	100	100	100	100	100	100	100	95	92	99%
15	GPT-5.4 Mini (Reasoning)	$0.013	18.6s	97%	100	100	100	100	100	100	97	97	97	97	99%
70	Claude Opus 4.5	$0.050	8.2s	95%	100	100	100	100	100	97	97	97	97	97	98%
34	Aion 2.0	$0.0095	1.1m	95%	100	100	100	100	100	97	97	97	97	97	98%
5	Stealth: Healer Alpha	$0.0000	16.1s	95%	100	100	100	100	100	100	100	97	93	93	98%
62	ByteDance Seed 2.0 Mini	$0.0030	2.6m	95%	100	100	100	100	100	100	97	97	97	92	98%
56	o4 Mini High	$0.031	45.4s	94%	100	100	100	100	100	97	97	97	95	95	98%
9	Gemini 2.5 Flash (Reasoning)	$0.0072	10.7s	94%	100	100	100	100	100	100	97	95	95	92	98%
81	Qwen 3.5 397B A17B	$0.023	2.2m	93%	100	100	100	100	100	100	100	95	92	92	98%
32	o4 Mini	$0.016	25.1s	93%	100	100	100	100	100	97	97	97	97	92	98%
24	Z.AI GLM 4.7	$0.0070	28.9s	92%	100	100	100	100	100	100	100	97	92	88	98%
71	GPT-5.1	$0.034	49.6s	92%	100	100	100	100	97	97	97	97	95	93	98%
101	GPT-5	$0.056	1.4m	93%	100	100	100	100	100	100	97	95	92	92	98%
21	Gemini 3 Flash (Preview, Reasoning)	$0.0093	14.7s	92%	100	100	100	100	100	100	100	92	92	92	98%
76	Gemini 3 Pro (Preview)	$0.045	22.7s	92%	100	100	100	100	100	100	100	92	92	92	98%
26	Qwen 3.5 Flash	$0.0036	47.1s	92%	100	100	100	100	100	100	100	92	92	92	98%
27	Z.AI GLM 5	$0.0082	33.0s	92%	100	100	100	100	100	97	97	97	92	92	97%
65	Qwen 3.5 122B	$0.025	1.1m	93%	100	100	100	100	100	100	97	92	92	92	97%
14	Xiaomi MIMO v2.5	$0.0045	16.2s	93%	100	100	100	100	100	100	97	92	92	92	97%
69	Nemotron 3 Super	$0.0000	2.8m	93%	100	100	97	97	97	97	97	97	97	93	97%
39	ByteDance Seed 1.6	$0.0067	53.0s	91%	100	100	100	100	100	100	97	92	92	88	97%
12	Gemini 2.5 Flash Lite (Reasoning)	$0.0016	11.6s	91%	100	100	100	97	97	97	97	97	93	90	97%
23	ByteDance Seed 2.0 Lite	$0.0060	51.4s	97%	97	97	97	97	97	97	97	97	97	97	97%
20	Stealth: Hunter Alpha	$0.0000	32.5s	91%	100	100	100	100	100	97	93	92	92	92	97%
50	GPT-5.2	$0.023	22.0s	90%	100	100	100	95	95	95	95	95	95	91	96%
49	Claude Sonnet 4	$0.029	7.4s	94%	97	97	97	97	97	97	97	93	93	93	96%
19	Gemini 2.5 Flash	$0.0027	3.3s	90%	100	100	97	97	97	97	95	95	91	87	96%
36	Mistral Large 2	$0.016	6.3s	92%	100	97	97	97	97	97	93	93	93	92	96%
61	Claude Sonnet 4.6	$0.030	7.7s	90%	97	97	97	97	97	97	97	97	88	88	95%
60	Claude 3.7 Sonnet	$0.030	8.2s	90%	97	97	97	97	97	97	97	97	88	88	95%
82	Qwen 3.5 9B	$0.0020	2.0m	80%	100	100	100	100	97	97	97	97	92	70	95%
42	Z.AI GLM 4.5 Air	$0.0027	36.8s	87%	100	100	97	97	93	93	92	92	92	90	95%
25	Inception Mercury 2	$0.0029	4.4s	88%	100	97	97	97	97	95	92	91	88	88	94%
43	Mistral Small 4 (Reasoning)	$0.0020	15.1s	83%	100	97	97	97	97	93	93	93	93	76	94%
31	Mistral Large 3	$0.0040	6.6s	87%	100	97	97	97	97	93	92	88	88	88	94%
38	GPT-5.4 Mini (Reasoning, Low)	$0.0054	6.2s	86%	97	97	97	97	97	97	97	92	83	83	94%
80	GPT-OSS 120B	$0.0024	1.9m	83%	100	100	97	97	97	93	93	88	88	79	93%
95	Qwen 3.6 27B	$0.019	1.1m	73%	100	100	100	100	100	100	100	97	73	61	93%
73	GPT-4o, May 13th (temp=0)	$0.038	3.4s	91%	93	93	93	93	93	93	93	93	90	90	93%
51	Mistral Large	$0.016	6.2s	86%	97	97	97	97	97	97	93	84	84	84	93%
45	MiniMax M2.7	$0.0021	24.4s	86%	100	97	93	93	93	93	92	90	88	84	92%
47	DeepSeek V4 Pro	$0.0041	19.9s	85%	100	95	92	92	92	92	92	92	92	83	92%
57	Stealth: Aurora Alpha	—	5.8s	83%	97	97	97	97	97	93	92	88	88	75	92%
131	Claude Opus 4	$0.144	11.2s	82%	100	100	97	92	92	92	92	88	84	83	92%
53	Qwen 3.5 Plus (2026-02-15)	$0.0041	20.6s	82%	100	100	92	92	92	92	92	87	83	83	91%
111	Claude Opus 4.7	$0.066	6.0s	79%	100	100	100	92	92	92	92	83	83	79	91%
48	Gemini 3 Flash (Preview)	$0.0032	4.1s	83%	100	92	92	92	92	92	92	83	83	83	90%
135	MoonshotAI: Kimi K2.6	$0.035	3.6m	42%	100	100	100	100	100	100	100	100	95	3	90%
91	Z.AI GLM 4.6	$0.0035	46.7s	69%	100	100	100	100	92	92	91	88	75	61	90%
59	MiniMax M2.5	$0.0021	26.0s	81%	97	97	93	93	93	90	90	84	80	79	90%
79	Z.AI GLM 4.7 Flash	$0.0017	1.0m	79%	97	97	93	92	92	90	90	84	80	76	89%
46	Gemini 3.1 Flash Lite	$0.0021	2.0s	85%	92	92	92	92	92	92	87	83	83	83	89%
122	DeepSeek V4 Pro (Reasoning)	$0.0098	1.8m	41%	100	100	100	100	100	100	95	95	95	0	89%
107	Claude 3.5 Sonnet	$0.056	9.4s	81%	97	93	88	88	88	88	88	84	84	84	88%
67	DeepSeek V3.2	$0.0019	17.7s	79%	95	93	93	92	92	88	84	80	79	79	88%
58	Gemini 3.1 Flash Lite (Reasoning)	$0.0021	7.2s	80%	95	92	92	92	92	88	83	83	79	79	88%
64	Gemini 3.1 Flash Lite (Preview)	$0.0024	1.9s	79%	97	91	88	88	88	88	88	84	83	75	87%
77	DeepSeek-V2 Chat	$0.0027	10.9s	73%	97	97	93	92	92	92	90	75	70	70	87%
100	GPT-5 Nano	$0.0044	1.6m	74%	97	95	93	87	84	84	84	84	83	75	87%
83	GPT-4o, Aug. 6th (temp=1)	$0.013	3.1s	76%	93	93	93	93	90	88	84	80	80	70	87%
72	ByteDance Seed 1.6 Flash	$0.0010	11.5s	76%	93	93	93	90	90	87	87	84	76	72	87%
117	Nemotron 3 Nano	$0.0030	4.2m	76%	97	93	90	88	88	87	87	84	76	75	86%
84	GPT-4o, Aug. 6th (temp=0)	$0.012	2.8s	76%	93	93	90	90	90	84	84	84	80	70	86%
75	Ministral 3 14B	$0.0014	6.2s	75%	93	92	90	88	88	87	83	83	72	71	85%
89	Grok 4.20	$0.0084	5.2s	72%	95	92	92	91	87	83	79	79	76	71	84%
86	DeepSeek V3 (2024-12-26)	$0.0028	13.9s	72%	97	92	92	84	84	83	80	79	76	75	84%
108	GPT-4o, May 13th (temp=1)	$0.038	3.7s	74%	90	90	88	87	84	80	80	80	76	75	83%
88	Qwen3 235B A22B Instruct 2507	$0.0010	22.8s	73%	93	92	88	84	83	83	79	75	75	75	83%
93	DeepSeek V3 (2025-03-24)	$0.0020	9.9s	69%	93	92	88	87	83	79	75	75	74	71	82%
115	Gemma 4 26B	$0.0008	16.7s	42%	92	92	92	92	92	92	92	92	83	0	82%
85	GPT-4.1 Mini	$0.0015	4.4s	73%	90	84	84	84	80	80	79	79	76	75	81%
102	DeepSeek V3.1	$0.0018	30.1s	65%	97	92	87	84	83	83	83	79	62	62	81%
104	GPT-4.1	$0.0092	9.9s	65%	100	88	83	83	79	79	79	79	75	62	81%
87	Gemma 3 27B	$0.0007	10.2s	74%	87	87	84	80	80	80	80	79	75	75	81%
99	Writer: Palmyra X5	$0.0072	17.3s	68%	90	88	88	84	79	79	79	76	75	67	81%
118	Claude Haiku 4.5	$0.010	5.0s	42%	97	97	92	88	88	88	88	83	79	3	80%
98	Grok 4.20 (Beta)	$0.0075	2.2s	67%	88	87	87	87	79	76	75	75	71	71	80%
97	Mistral Medium 3.1	$0.0038	7.9s	67%	92	91	83	83	83	76	75	75	68	67	79%
96	DeepSeek V4 Flash	$0.0004	8.0s	65%	87	87	83	83	83	79	79	79	75	52	79%
103	Hermes 3 405B	$0.0067	12.8s	69%	83	83	83	83	80	75	72	71	70	70	77%
109	Z.AI GLM 4.5	$0.0025	12.4s	58%	93	88	88	88	80	79	76	75	53	50	77%
112	GPT-5.4 Mini	$0.0037	1.9s	53%	95	95	95	91	87	80	66	52	52	52	76%
106	Inception Mercury	$0.0006	10.9s	61%	92	84	79	79	75	75	71	71	67	54	75%
105	Mistral Small Creative	$0.0008	3.6s	63%	84	83	83	79	79	79	75	67	61	52	74%
110	Ministral 3 3B	$0.0007	1.7s	59%	83	83	83	80	75	72	71	67	60	53	73%
125	Qwen 3 32B	$0.0014	23.3s	39%	97	90	87	80	79	79	71	61	55	3	70%
113	Qwen 2.5 72B	$0.0011	8.6s	59%	83	80	76	76	72	72	66	61	61	53	70%
124	Grok 4.3	$0.0077	5.2s	41%	88	83	83	79	75	75	72	71	68	3	70%
121	Ministral 3 8B	$0.0011	3.5s	41%	84	84	84	83	83	79	67	67	63	0	69%
114	Mistral Small 4	$0.0008	3.4s	56%	79	79	76	75	72	72	66	60	56	44	68%
116	Mistral Small 3.2 24B	$0.0008	6.5s	55%	80	75	75	75	71	66	63	63	63	44	68%
126	Llama 3.1 Nemotron 70B	$0.0085	20.0s	48%	84	83	75	75	66	66	62	53	49	40	65%
120	Ministral 3B	$0.0003	1.3s	48%	83	80	72	66	60	60	60	55	55	55	65%
119	WizardLM 2 8x22b	$0.0051	8.0s	54%	76	75	75	66	66	66	62	55	53	51	64%
123	Hermes 3 70B	$0.0020	23.5s	50%	80	70	67	62	61	61	60	53	52	51	62%
129	Llama 3.1 70B	$0.0031	23.7s	35%	87	71	67	67	63	62	57	53	50	0	58%
127	Arcee AI: Trinity Mini	$0.0004	6.1s	42%	77	73	67	61	61	53	53	44	44	33	57%
146	Arcee AI: Trinity Large (Preview)	$0.0000	8.8m	35%	87	79	75	71	61	58	34	34	33	33	57%
136	GPT-5.4 Nano (Reasoning, Low)	$0.0011	4.9s	16%	93	92	92	88	88	61	41	0	0	0	55%
128	GPT-4o Mini (temp=0)	$0.0008	8.7s	42%	66	61	57	57	48	48	48	48	48	42	52%
134	Cohere Command R+ (Aug. 2024)	$0.020	10.4s	42%	66	66	60	60	60	48	44	41	40	33	52%
130	GPT-4o Mini (temp=1)	$0.0007	7.8s	41%	60	55	55	48	48	48	48	44	40	33	48%
143	GPT-5.4 Nano (Reasoning)	$0.0026	13.6s	4%	97	97	97	83	67	3	0	0	0	0	44%
132	Gemma 3 12B	$0.0003	7.1s	39%	55	48	48	48	48	40	40	40	40	33	44%
137	Gemini 2.5 Flash Lite	$0.0008	3.3s	28%	66	65	59	58	56	54	49	18	0	0	43%
133	Claude 3 Haiku	$0.0021	3.2s	40%	44	44	44	42	42	42	40	40	40	39	42%
140	GPT-5.4 Nano	$0.0008	3.2s	18%	83	80	66	54	54	48	3	0	0	0	39%
139	Mistral NeMO	$0.0011	6.2s	24%	72	48	41	38	37	36	35	33	33	0	37%
138	Gemma 3 4B	$0.0003	5.0s	33%	42	40	39	38	37	33	33	33	33	33	36%
144	Ministral 8B	$0.0007	2.2s	10%	75	75	71	67	67	0	0	0	0	0	35%
141	Llama 3.1 8B	$0.0002	36.7s	23%	60	57	44	42	38	33	32	25	3	3	34%
142	Rocinante 12B	$0.0015	5.4s	23%	44	42	38	33	33	33	33	17	3	0	28%
147	LFM2 24B	$0.0019	5.0m	25%	35	34	33	33	33	33	33	32	3	3	27%
145	GPT-4.1 Nano	$0.0003	2.2s	14%	33	18	17	17	17	17	17	17	0	0	15%
84.46%

Median	Evaluator	Top 3	Flop 3
84.5%	Accuracy (recall)	100Gemini 3.1 Pro (Preview) 100Gemma 4 31B 100GPT-5.5	0LFM2 24B 0GPT-4.1 Nano 0Gemma 3 4B
94.7%	Precision	100Grok 4 Fast 100Claude Sonnet 4.6 (Reasoning) 100Qwen 3.6 35B	1GPT-4.1 Nano 1LFM2 24B 7Rocinante 12B
100.0%	Structural validity	100Qwen 3.6 35B 100GPT-5 100GPT-5.4 Mini	45GPT-4.1 Nano 50Ministral 8B 51GPT-5.4 Nano (Reasoning)

Codex Violation Detection

40 codex entries

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)