Codex Violation Detection

Detects factual inconsistencies between a story bible and prose passages. The model must output structured XML identifying each violation with paragraph number and substring.

Price-Performance Score Distribution (Top 20)

Click a model name to view its detail page.

ScoreCostTime
Z.AI GLM 5.3 Flash (Reasoning, Low)96%$0.000416.9s
GPT-6 Sol99%$0.00746.5s
Muse Ember (Reasoning, Max)98%$0.004112.5s
Xiaomi MIMO v2.6 Flash96%$0.000639.8s
Qwen 3.7 Flash (Reasoning)97%$0.000933.3s
DeepSeek V4.1 Flash 1017 (Reasoning, High)98%$0.002234.1s
DeepSeek V4.1 Flash (Reasoning, High)98%$0.004630.8s
GPT-5.6 Terra98%$0.0113.8s
Xiaomi MIMO v2.6 Pro98%$0.002340.3s
Gemini 4 Argon (Reasoning, Medium)99%$0.0137.2s
Gemini 3.7 Flash (Reasoning, Medium)99%$0.0139.9s
Z.AI GLM 5 Turbo97%$0.007217.1s
GPT-5.497%$0.0138.8s
Gemini 2.5 Flash (Reasoning)97%$0.007912.5s
GPT-6 Sol (Reasoning, Medium)98%$0.01313.9s
GPT-6 Luna (Reasoning, Medium)96%$0.001018.7s
Gemini 3.8 Flash (Reasoning, Medium)100%$0.02013.0s
GPT-6 Luna (Reasoning, High)96%$0.001422.5s
GPT-6 Luna93%$0.00046.0s
Z.AI GLM 5.2 (Reasoning, High)97%$0.010027.9s
0.700.800.901.00

Cost vs Performance

Compares total cost for this test against the test score. Quadrant lines are drawn at the median values. Only models with available cost data are shown.

22 low-scoring outliers hidden: Llama 3.1 70B (71.9%), Arcee AI: Trinity Mini (71.4%), Ministral 3 14B (70.8%), Gemini 2.5 Flash Lite (67.0%), Gemma 3 12B (65.6%), Ministral 3 8B (65.2%), GPT-4o Mini (temp=0) (64.9%), GPT-4o Mini (temp=1) (64.9%), Cydonia 24B V4.1 (64.3%), Mistral Small 4 (62.0%), Ministral 3 3B (61.5%), Cohere Command R+ (Aug. 2024) (61.4%), Hermes 3 70B (60.5%), Ministral 8B (59.6%), Ministral 3B (59.5%), Mistral NeMO (53.6%), Aion 3.0 Mini (47.3%), GPT-5.4 Nano (Reasoning, Low) (42.6%), Gemma 3 4B (39.2%), WizardLM 2 8x22b (36.9%), GPT-5.4 Nano (36.6%), GPT-4.1 Nano (21.9%).

Top Overall Models (Top 20)

Ranked by composite score (performance, cost, speed & stability). Click a model name to view its detail page.

ScoreCostSpeedStability
GPT-6 Sol99%$0.00746.5s97%
Muse Ember (Reasoning, Max)98%$0.004112.5s95%
Gemini 4 Argon (Reasoning, Medium)99%$0.0137.2s97%
Gemini 3.7 Flash (Reasoning, Medium)99%$0.0139.9s97%
GPT-5.6 Terra98%$0.0113.8s94%
GPT-6.1 Sol (Reasoning, Medium)98%$0.009113.4s94%
Qwen 3.7 Flash (Reasoning)97%$0.000933.3s93%
Z.AI GLM 5.3 Flash (Reasoning, Low)96%$0.000416.9s89%
Gemini 2.5 Flash (Reasoning)97%$0.007912.5s92%
DeepSeek V4.1 Flash 1017 (Reasoning, High)98%$0.002234.1s93%
Z.AI GLM 5 Turbo97%$0.007217.1s92%
Gemini 4 Hydrogen (Reasoning, Low)100%$0.01814.4s98%
Xiaomi MIMO v2.6 Pro98%$0.002340.3s94%
GPT-6.1 Terra (Reasoning, Medium)96%$0.005411.3s90%
DeepSeek V4.1 Flash (Reasoning, High)98%$0.004630.8s93%
GPT-6 Luna (Reasoning, Medium)96%$0.001018.7s88%
GPT-6 Sol (Reasoning, Medium)98%$0.01313.9s94%
Gemini 3.8 Flash (Reasoning, Medium)100%$0.02013.0s98%
Muse Spark 1.1 (Reasoning, Minimal)97%$0.01110.5s92%
GPT-5.497%$0.0138.8s91%
80%90%100%
matrixtiers
Model Total â–¼Small codex (7 entries), short passage (165 words)Large codex (40 entries), short passage (165 words)Small codex (7 entries), long passage (734 words)Large codex (40 entries), long passage (1,019 words)5 codex entries10 codex entries20 codex entries40 codex entries
Gemini 4 Neon (Reasoning, Medium)100%100%100%100%98%100%100%100%100%
Gemini 4 Carbon (Reasoning, Medium)100%100%100%100%98%100%100%100%100%
Gemini 4 Hydrogen (Reasoning, Medium)100%100%100%99%98%100%100%100%100%
Gemini 4 Neon (Reasoning, Low)100%100%100%99%98%100%100%100%100%
Gemini 4 Hydrogen (Reasoning, Low)100%100%100%100%99%100%100%98%100%
Gemini 3.8 Flash (Reasoning, Medium)100%100%100%99%98%100%100%100%100%
Claude Opus 5.5 (Reasoning)99%100%99%99%98%100%100%100%100%
Claude Opus 4.599%100%100%100%96%100%100%100%98%
Gemini 4 Argon (Reasoning, Medium)99%100%97%100%98%100%100%99%100%
Gemini 3.1 Pro (Preview)99%100%99%99%96%100%100%99%100%
Kimi K3 (Reasoning, High)99%100%99%98%98%100%100%98%100%
GPT-6 Sol99%100%100%99%97%100%100%97%100%
Gemini 3.7 Flash (Reasoning, Medium)99%100%99%97%96%100%100%100%100%
Z.AI GLM 5.3 (Reasoning, Max)99%100%98%98%99%99%100%98%100%
Ember-1 (Reasoning, Max)99%100%98%98%98%100%100%97%100%
1–15 of 212
Page 1 / 15

matrix

Small codex (7 entries), short passage (165 words)

Large codex (40 entries), short passage (165 words)

Small codex (7 entries), long passage (734 words)

Large codex (40 entries), long passage (1,019 words)

tiers

5 codex entries

10 codex entries

20 codex entries

40 codex entries