Hallucination
28 scenarios across 3 subcategories. 155 models scored.
Subcategories
| Subcategory | Avg Score | Best Model | Best Score |
|---|---|---|---|
| False Positives | 57.58% | Nemotron 3 Super | 99.06% |
| Content Invention | 96.74% | Qwen3.7 Max | 100.00% |
| Output Corruption | 99.13% | Claude Opus 4.6 (Reasoning) | 100.00% |
Model Leaderboard
All models ranked by their Hallucination category score.
| # | Model | Hallucination | False Positives | Content Invention | Output Corruption | Overall |
|---|---|---|---|---|---|---|
| 1 | Stealth: Aurora Alpha | 99.93% | – | – | 99.93% | 83.79% |
| 2 | Nemotron 3 Super | 99.69% | 99.06% | 100.00% | 100.00% | 84.56% |
| 3 | o4 Mini High | 99.06% | 97.19% | 100.00% | 100.00% | 90.29% |
| 4 | GPT-5.4 Nano (Reasoning, Low) | 99.03% | 97.08% | 100.00% | 100.00% | 79.48% |
| 5 | Grok 4.1 Fast | 99.02% | 97.08% | 100.00% | 99.99% | 89.55% |
| 6 | o4 Mini | 98.75% | 96.25% | 100.00% | 100.00% | 88.35% |
| 7 | Z.AI GLM 5 Turbo | 98.64% | 95.94% | 100.00% | 99.99% | 94.27% |
| 8 | GPT-5.1 | 98.44% | 95.31% | 100.00% | 100.00% | 92.54% |
| 9 | GPT-5.4 Mini (Reasoning, Low) | 98.43% | 95.31% | 100.00% | 99.98% | 85.75% |
| 10 | Claude Opus 4.6 (Reasoning) | 98.13% | 94.38% | 100.00% | 100.00% | 95.02% |
| 11 | GPT-5.4 Nano (Reasoning) | 97.95% | 93.85% | 100.00% | 100.00% | 81.36% |
| 12 | Z.AI GLM 5 | 97.74% | 93.23% | 100.00% | 100.00% | 91.23% |
| 13 | GPT-5 Mini | 97.71% | 93.12% | 100.00% | 100.00% | 92.62% |
| 14 | Z.AI GLM 5.1 | 97.67% | 93.02% | 100.00% | 100.00% | 94.37% |
| 15 | Gemma 4 26B (Reasoning) | 97.59% | 92.81% | 100.00% | 99.95% | 91.49% |
| 16 | Grok 4.3 (Reasoning) | 97.40% | 92.19% | 100.00% | 100.00% | 93.60% |
| 17 | Claude Opus 4.7 (Reasoning) | 97.39% | 92.19% | 100.00% | 99.98% | 93.23% |
| 18 | Qwen 3.5 Plus (2026-04-20) | 97.13% | 92.50% | 100.00% | 98.89% | 91.51% |
| 19 | MiniMax M2.7 | 96.47% | 89.69% | 100.00% | 99.72% | 89.10% |
| 20 | Grok 4.20 (Beta, Reasoning) | 96.28% | 88.85% | 100.00% | 100.00% | 91.49% |
| 21 | GPT-5.4 Mini (Reasoning) | 96.22% | 88.65% | 100.00% | 100.00% | 90.65% |
| 22 | Qwen 3.6 27B | 95.73% | 87.24% | 100.00% | 99.95% | 89.72% |
| 23 | MiniMax M3 | 95.66% | 86.98% | 100.00% | 100.00% | 90.88% |
| 24 | Grok 4.20 (Reasoning) | 95.66% | 86.98% | 100.00% | 100.00% | 91.39% |
| 25 | Gemini 2.5 Flash (Reasoning) | 95.60% | 86.87% | 100.00% | 99.94% | 86.51% |
| 26 | Gemini 2.5 Flash Lite (Reasoning) | 95.59% | 91.56% | 95.24% | 99.96% | 85.75% |
| 27 | GPT-OSS 120B | 95.29% | 85.94% | 100.00% | 99.92% | 86.44% |
| 28 | Xiaomi MIMO v2.5 | 95.25% | 85.83% | 100.00% | 99.93% | 85.05% |
| 29 | GPT-4.1 | 95.24% | 85.76% | 100.00% | 99.97% | 88.68% |
| 30 | Xiaomi MIMO v2.5 Pro | 95.17% | 85.52% | 100.00% | 100.00% | 87.36% |
| 31 | ByteDance Seed 1.6 | 95.14% | 85.42% | 100.00% | 100.00% | 90.70% |
| 32 | GPT-5.2 | 95.09% | 85.28% | 100.00% | 100.00% | 90.26% |
| 33 | Inception Mercury | 95.08% | 98.13% | 92.86% | 94.25% | 79.50% |
| 34 | DeepSeek V4 Flash (Reasoning) | 95.02% | 88.65% | 96.43% | 100.00% | 89.01% |
| 35 | Claude Opus 4.8 (Reasoning) | 94.83% | 84.48% | 100.00% | 100.00% | 92.22% |
| 36 | Stealth: Healer Alpha | 94.67% | 84.06% | 100.00% | 99.93% | 85.93% |
| 37 | Claude Opus 4.8 (Reasoning, Low) | 94.65% | 83.96% | 100.00% | 100.00% | 92.14% |
| 38 | ByteDance Seed 1.6 Flash | 94.52% | 94.11% | 89.88% | 99.57% | 73.27% |
| 39 | Gemma 4 31B (Reasoning) | 94.41% | 83.23% | 100.00% | 100.00% | 91.71% |
| 40 | Qwen3.7 Max | 94.32% | 83.02% | 100.00% | 99.95% | 95.75% |
| 41 | Claude Sonnet 4.6 (Reasoning) | 93.96% | 81.88% | 100.00% | 100.00% | 93.66% |
| 42 | Claude Opus 4.6 | 93.60% | 80.81% | 100.00% | 100.00% | 92.35% |
| 43 | Aion 2.0 | 93.57% | 87.81% | 92.89% | 100.00% | 89.21% |
| 44 | GPT-5 Nano | 93.52% | 92.50% | 88.10% | 99.96% | 82.60% |
| 45 | Mistral Small 4 (Reasoning) | 92.98% | 82.50% | 96.43% | 100.00% | 82.39% |
| 46 | MiniMax M2.5 | 92.94% | 79.32% | 100.00% | 99.50% | 88.71% |
| 47 | Z.AI GLM 4.5 Air | 92.74% | 79.34% | 100.00% | 98.87% | 83.12% |
| 48 | Inception Mercury 2 | 92.60% | 96.25% | 81.55% | 99.99% | 83.85% |
| 49 | Ministral 3 8B | 92.52% | 77.55% | 100.00% | 100.00% | 71.76% |
| 50 | GPT-5.4 (Reasoning, Low) | 92.29% | 76.88% | 100.00% | 100.00% | 91.41% |
| 51 | Qwen 3.6 Flash | 92.26% | 76.77% | 100.00% | 100.00% | 90.65% |
| 52 | GPT-5 | 91.84% | 75.52% | 100.00% | 100.00% | 91.93% |
| 53 | Qwen3.6 Max Preview | 91.72% | 75.16% | 100.00% | 100.00% | 94.54% |
| 54 | ByteDance Seed 2.0 Mini | 91.29% | 73.98% | 100.00% | 99.90% | 86.91% |
| 55 | Gemini 3.5 Flash (Reasoning) | 91.22% | 73.65% | 100.00% | 100.00% | 94.08% |
| 56 | Arcee AI: Trinity Mini | 91.12% | 73.37% | 100.00% | 100.00% | 70.90% |
| 57 | Grok 4 Fast | 91.09% | 73.28% | 100.00% | 100.00% | 86.15% |
| 58 | DeepSeek V4 Pro (Reasoning) | 90.88% | 73.75% | 100.00% | 98.89% | 90.10% |
| 59 | Stealth: Hunter Alpha | 90.78% | 72.33% | 100.00% | 100.00% | 87.34% |
| 60 | LFM2 24B | 90.53% | 71.58% | 100.00% | 100.00% | 58.77% |
| 61 | GPT-5.4 (Reasoning) | 90.43% | 71.30% | 100.00% | 100.00% | 93.24% |
| 62 | Qwen 3.6 35B | 90.19% | 71.67% | 100.00% | 98.89% | 89.05% |
| 63 | Gemma 4 31B | 90.17% | 70.52% | 100.00% | 100.00% | 86.91% |
| 64 | Z.AI GLM 4.6 | 90.08% | 71.35% | 100.00% | 98.89% | 89.11% |
| 65 | Claude Sonnet 4.6 | 89.98% | 69.98% | 100.00% | 99.97% | 91.15% |
| 66 | Z.AI GLM 4.7 Flash | 89.86% | 76.72% | 92.86% | 100.00% | 84.82% |
| 67 | Qwen 3 32B | 89.56% | 69.27% | 100.00% | 99.39% | 82.21% |
| 68 | Grok 4 | 89.45% | 68.35% | 100.00% | 100.00% | 88.12% |
| 69 | GPT-5.4 Nano | 89.29% | 67.86% | 100.00% | 100.00% | 74.40% |
| 70 | Ministral 8B | 89.19% | 67.57% | 100.00% | 100.00% | 64.87% |
| 71 | Nemotron 3 Nano | 89.16% | 87.60% | 82.14% | 97.72% | 77.73% |
| 72 | Gemini 3.1 Pro (Preview) | 89.06% | 67.19% | 100.00% | 100.00% | 94.37% |
| 73 | MoonshotAI: Kimi K2.6 | 88.85% | 66.56% | 100.00% | 100.00% | 92.31% |
| 74 | Z.AI GLM 4.7 | 88.47% | 65.47% | 100.00% | 99.96% | 88.69% |
| 75 | Gemma 4 26B | 88.45% | 65.34% | 100.00% | 100.00% | 85.84% |
| 76 | Gemini 3 Pro (Preview) | 88.23% | 64.69% | 100.00% | 100.00% | 88.79% |
| 77 | MoonshotAI: Kimi K2.5 | 88.02% | 64.06% | 100.00% | 100.00% | 91.04% |
| 78 | GPT-4.1 Nano | 87.73% | 63.25% | 100.00% | 99.95% | 71.94% |
| 79 | Z.AI GLM 4.5 | 87.05% | 62.31% | 100.00% | 98.85% | 86.27% |
| 80 | Qwen 3.5 122B | 86.58% | 59.74% | 100.00% | 100.00% | 91.53% |
| 81 | Qwen 3.5 27B | 86.58% | 59.73% | 100.00% | 100.00% | 90.85% |
| 82 | Gemini 2.5 Pro | 86.11% | 58.32% | 100.00% | 100.00% | 88.53% |
| 83 | Claude Opus 4.7 | 85.89% | 57.68% | 100.00% | 100.00% | 89.93% |
| 84 | Qwen 3.5 9B | 85.58% | 62.64% | 96.43% | 97.67% | 86.05% |
| 85 | Gemini 3 Flash (Preview, Reasoning) | 85.32% | 55.98% | 100.00% | 99.97% | 90.50% |
| 86 | GPT-5.5 (Reasoning, Low) | 84.41% | 53.23% | 100.00% | 100.00% | 92.59% |
| 87 | GPT-5.5 (Reasoning) | 84.24% | 52.71% | 100.00% | 100.00% | 92.98% |
| 88 | Claude Haiku 4.5 | 83.86% | 51.59% | 100.00% | 100.00% | 85.14% |
| 89 | Qwen 3.5 397B A17B | 82.10% | 46.35% | 100.00% | 99.93% | 91.73% |
| 90 | Mistral Medium 3.1 | 82.09% | 46.27% | 100.00% | 100.00% | 77.83% |
| 91 | Claude Opus 4.5 | 82.06% | 46.19% | 100.00% | 100.00% | 89.69% |
| 92 | Cydonia 24B V4.1 | 81.53% | 55.90% | 91.07% | 97.62% | 75.09% |
| 93 | GPT-4.1 Mini | 81.14% | 43.42% | 100.00% | 100.00% | 83.20% |
| 94 | Qwen 3.5 35B | 80.87% | 49.74% | 92.86% | 100.00% | 88.00% |
| 95 | Qwen 3.5 Flash | 80.63% | 49.01% | 92.90% | 99.98% | 86.38% |
| 96 | GPT-5.5 | 80.57% | 41.70% | 100.00% | 100.00% | 89.09% |
| 97 | Claude Sonnet 4 | 80.08% | 40.23% | 100.00% | 100.00% | 88.72% |
| 98 | Ministral 3 14B | 79.99% | 39.97% | 100.00% | 100.00% | 72.54% |
| 99 | ByteDance Seed 2.0 Lite | 79.75% | 41.46% | 100.00% | 97.78% | 84.80% |
| 100 | Hermes 3 405B | 79.70% | 39.09% | 100.00% | 100.00% | 82.86% |
| 101 | Qwen 2.5 72B | 79.56% | 40.41% | 100.00% | 98.28% | 75.46% |
| 102 | GPT-4o, Aug. 6th (temp=1) | 79.53% | 38.76% | 100.00% | 99.82% | 82.62% |
| 103 | Gemini 3.5 Flash (Reasoning, Minimal) | 79.09% | 37.28% | 100.00% | 100.00% | 86.47% |
| 104 | DeepSeek V4 Pro | 78.72% | 36.18% | 100.00% | 99.98% | 82.63% |
| 105 | GPT-5.4 Mini | 78.40% | 35.21% | 100.00% | 100.00% | 82.43% |
| 106 | Grok 4.20 (Beta) | 78.28% | 34.84% | 100.00% | 100.00% | 83.85% |
| 107 | Mistral Large 3 | 78.17% | 34.52% | 100.00% | 100.00% | 85.43% |
| 108 | Mistral Large 2 | 77.87% | 33.60% | 100.00% | 100.00% | 82.41% |
| 109 | Mistral Large | 77.50% | 32.50% | 100.00% | 100.00% | 80.15% |
| 110 | Grok 4.3 | 77.16% | 31.51% | 100.00% | 99.96% | 78.66% |
| 111 | GPT-4o Mini (temp=1) | 76.78% | 30.35% | 100.00% | 100.00% | 79.08% |
| 112 | Arcee AI: Trinity Large (Preview) | 76.47% | 29.41% | 100.00% | 100.00% | 73.33% |
| 113 | Claude 3.5 Sonnet | 76.31% | 28.94% | 100.00% | 100.00% | 84.24% |
| 114 | Gemini 2.5 Flash Lite | 76.17% | 28.56% | 100.00% | 99.96% | 81.08% |
| 115 | Gemini 3.1 Flash Lite | 75.72% | 27.15% | 100.00% | 100.00% | 85.75% |
| 116 | Claude Opus 4 | 75.68% | 27.04% | 100.00% | 100.00% | 87.69% |
| 117 | Mistral Small 3.2 24B | 75.64% | 28.78% | 100.00% | 98.14% | 78.58% |
| 118 | Claude Sonnet 4.5 | 75.57% | 26.70% | 100.00% | 100.00% | 88.03% |
| 119 | DeepSeek V4 Flash | 75.47% | 26.40% | 100.00% | 100.00% | 82.02% |
| 120 | Gemini 3.1 Flash Lite (Reasoning) | 75.26% | 25.79% | 100.00% | 100.00% | 86.41% |
| 121 | Claude 3.7 Sonnet | 75.18% | 25.54% | 100.00% | 100.00% | 83.39% |
| 122 | Llama 3.1 Nemotron 70B | 74.99% | 28.79% | 96.43% | 99.75% | 74.70% |
| 123 | Gemini 3.1 Flash Lite (Preview) | 74.58% | 23.78% | 100.00% | 99.97% | 85.87% |
| 124 | Mistral Small Creative | 74.46% | 23.38% | 100.00% | 100.00% | 73.27% |
| 125 | GPT-5.4 | 73.78% | 21.34% | 100.00% | 100.00% | 84.32% |
| 126 | GPT-4o Mini (temp=0) | 73.56% | 20.67% | 100.00% | 100.00% | 78.29% |
| 127 | Mistral Small 4 | 73.41% | 20.22% | 100.00% | 100.00% | 76.46% |
| 128 | GPT-4o, May 13th (temp=1) | 73.40% | 20.21% | 100.00% | 100.00% | 83.80% |
| 129 | GPT-4o, Aug. 6th (temp=0) | 73.35% | 20.06% | 100.00% | 100.00% | 82.45% |
| 130 | Qwen 3.5 Plus (2026-02-15) | 73.35% | 20.40% | 100.00% | 99.65% | 85.96% |
| 131 | DeepSeek V3 (2024-12-26) | 73.11% | 19.33% | 100.00% | 100.00% | 83.68% |
| 132 | Grok 4.20 | 73.01% | 19.05% | 100.00% | 99.99% | 81.70% |
| 133 | DeepSeek V3.1 | 72.80% | 18.52% | 100.00% | 99.89% | 82.39% |
| 134 | DeepSeek V3.2 | 72.50% | 17.51% | 100.00% | 100.00% | 82.25% |
| 135 | Writer: Palmyra X5 | 72.04% | 23.27% | 92.86% | 100.00% | 79.57% |
| 136 | Gemini 2.5 Flash | 71.70% | 16.03% | 100.00% | 99.08% | 80.60% |
| 137 | Gemini 3 Flash (Preview) | 71.24% | 13.77% | 100.00% | 99.95% | 85.35% |
| 138 | Ministral 3B | 70.75% | 26.52% | 85.71% | 100.00% | 61.29% |
| 139 | Ministral 3 3B | 70.45% | 22.08% | 89.29% | 100.00% | 67.22% |
| 140 | WizardLM 2 8x22b | 70.19% | 29.74% | 85.73% | 95.09% | 71.06% |
| 141 | Qwen3 235B A22B Instruct 2507 | 69.82% | 23.76% | 85.71% | 99.97% | 80.10% |
| 142 | Llama 3.1 70B | 69.78% | 29.26% | 92.86% | 87.24% | 78.40% |
| 143 | GPT-4o, May 13th (temp=0) | 69.76% | 9.29% | 100.00% | 100.00% | 85.36% |
| 144 | DeepSeek-V2 Chat | 69.48% | 17.97% | 90.48% | 100.00% | 84.83% |
| 145 | Gemma 3 12B | 69.15% | 7.45% | 100.00% | 100.00% | 78.41% |
| 146 | Gemma 3 27B | 68.74% | 6.23% | 100.00% | 100.00% | 77.85% |
| 147 | Gemma 3 4B | 67.60% | 2.80% | 100.00% | 100.00% | 68.57% |
| 148 | DeepSeek V3 (2025-03-24) | 67.07% | 18.55% | 88.10% | 94.58% | 81.99% |
| 149 | Hermes 3 70B | 67.03% | 42.95% | 60.79% | 97.34% | 72.57% |
| 150 | Skyfall 36B V2 | 66.29% | 44.87% | 56.07% | 97.93% | 65.76% |
| 151 | Cohere Command R+ (Aug. 2024) | 64.40% | 55.29% | 37.90% | 100.00% | 69.03% |
| 152 | Mistral NeMO | 62.63% | 11.73% | 76.19% | 99.97% | 65.04% |
| 153 | Claude 3 Haiku | 60.81% | 19.94% | 62.50% | 100.00% | 71.19% |
| 154 | Rocinante 12B | 52.07% | 27.40% | 38.95% | 89.87% | 54.54% |
| 155 | Llama 3.1 8B | 43.27% | 18.43% | 77.38% | 34.01% | 63.35% |