NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2026-09-24-01M39VVPF8EZE5BV51RR5C4M2R
Run Details
Model:
Aion 3.5 Mini (Reasoning, High)
Test:
Codex Violation Detection
Scenario:
10 codex entries
Duration:
1m 58s
Cost:
$0.007723
Provider:
AionLabs
Evaluation summary
Score
Evaluator
Details Link
100.00%
Accuracy (recall)
(details)
100.00%
Precision
(details)
100.00%
Structural validity
(details)
100.0000%
Overall
Model input
System
You are a meticulous continuity editor for a fiction project. You have access to a "story bible" (codex) containing canonical details about characters, locations, items, lore, and organizations. Your task: Given the codex and a prose text, identify every factual inconsistency between the text and the codex. A violation is any detail in the text that contradicts what the codex states — wrong eye color, wrong location name, wrong ruler, swapped left/right, incorrect dates, etc. Be thorough. Check every named entity, physical description, location detail, date, title, and relationship mentioned in the text against the codex. Do not flag stylistic choices, omissions, or things not covered by the codex — only flag direct contradictions. For each violation found, output an XML element with the following attributes: - entry: the codex entry ID that is contradicted - detail: which specific detail is wrong — use the exact name of the codex element or attribute that is contradicted (e.g. "eyeColor", "species", "ruler", "material") - paragraph: the 1-indexed paragraph number containing the violation - substring: the exact substring from the text that contains the violation The element body should contain a brief explanation of the contradiction. Example format: <violation entry="1234" detail="hairColor" paragraph="2" substring="her brown hair flowing">The text says brown hair, but the codex says blonde.</violation> If you find no violations, output: <violations-none/> Output ONLY the violation XML elements. Do not include any other text, preamble, or explanation outside the XML.
User/Human
<codex> <character id="4412" name="Aurora Carter" aliases="Rory, Laila, Aurora, Carter, Malphora" pronouns="she/her" species="Human" storyRole="Protagonist"> <eyeColor>bright blue</eyeColor> <hair>straight shoulder-length black</hair> <height>5'6"</height> <age>25</age> <scar>small crescent-shaped scar on her left wrist from a childhood accident</scar> <description>Aurora, referred to by her friends as Rory, is cool-headed, intelligent, and known for quick out-of-the-box thinking.</description> <backstory>Born in Cardiff to an Irish barrister father (Brendan Carter) and Welsh teacher mother (Jennifer Carter née Ellis), Rory reluctantly pursued Pre-Law at Cardiff University. She fled to London at the behest of her childhood friend Eva after leaving an abusive ex named Evan. By day, Rory works part-time as a delivery person for Yu-Fei Cheung's Golden Empress restaurant. She lives in a flat above Silas' bar.</backstory> </character> <character id="4413" name="Silas Blackwood" aliases="Silas, Si, the Spymaster" pronouns="he/him" species="Human" storyRole="Mentor"> <eyeColor>hazel</eyeColor> <hair>grey-streaked auburn</hair> <height>6'1"</height> <age>58</age> <beard>neatly trimmed, matching his grey-streaked auburn hair</beard> <limp>slight limp in his left leg from an old knee injury</limp> <signetRing>silver signet ring, always worn on his right hand</signetRing> <description>Silas is a retired intelligence operative turned bar owner. He carries himself with quiet authority.</description> <backstory>Former MI6 field agent who retired after a botched operation in Prague left him with a permanent knee injury. He opened "The Raven's Nest" bar in Soho as a front for his network of contacts.</backstory> </character> <character id="4414" name="Eva Kowalski" aliases="Eva, Evie" pronouns="she/her" species="Human" storyRole="Best Friend"> <eyeColor>green</eyeColor> <hair>curly red</hair> <height>5'4"</height> <age>26</age> <glasses>round glasses</glasses> <satchel>worn leather satchel full of books</satchel> <freckles>freckled complexion</freckles> <nervousHabit>tucking hair behind her left ear</nervousHabit> <description>Aurora's childhood friend and an occult researcher.</description> <backstory>Moved to London two years before Aurora and works as a research assistant at the British Museum's restricted archives. She holds a master's degree in Ancient History from Oxford.</backstory> </character> <character id="4415" name="Lucien Moreau" aliases="Lucien, Luc, the Frenchman" pronouns="he/him" species="Half-Demon" storyRole="Ally"> <eyeColor>heterochromatic — one amber, one black</eyeColor> <hair>slicked-back platinum blond</hair> <height>5'11"</height> <age>32</age> <cane>ivory-handled cane that conceals a thin blade</cane> <dress>tailored charcoal suits</dress> <description>An impeccably dressed fixer and information broker in London's supernatural underworld. He speaks four languages fluently.</description> <backstory>Born in Marseille to a human mother and a demon father from the realm of Avaros.</backstory> </character> <character id="4416" name="Detective Harlow Quinn" aliases="Quinn, Detective Quinn, Harlow" pronouns="she/her" species="Human" storyRole="Antagonist"> <eyeColor>brown</eyeColor> <hair>closely cropped salt-and-pepper</hair> <height>5'9"</height> <age>41</age> <jawline>sharp jaw</jawline> <watch>worn leather watch on her left wrist</watch> <bearing>military precision</bearing> <description>A Metropolitan Police detective with 18 years of decorated service who suspects the clique is involved in criminal activity.</description> <backstory>She lost her partner, DS Morris, during a case three years ago under unexplained circumstances that had supernatural origins she doesn't yet understand.</backstory> </character> <location id="853" name="Dymas (Gluttony)" aliases="Dymas, gluttony, Dymasian" realm="Hel" sin="Gluttony"> <ruler>Prince Belphegor</ruler> <skyColor>warm amber</skyColor> <description>A place of excess and indulgence. Sprawling vineyards, orchards, and gardens provide exotic ingredients for master chefs — often helbound souls contracted from Earth. Grand feasts and culinary competitions are common.</description> </location> <location id="854" name="The Raven's Nest" aliases="the Nest, Raven, Silas' bar" realm="Earth"> <signColor>distinctive green neon sign above the entrance</signColor> <secretRoom>hidden back room accessible through a bookshelf, used for clandestine meetings</secretRoom> <description>A dimly lit bar in Soho, London. The walls are covered with old maps and black-and-white photographs.</description> </location> <location id="855" name="Avaros (Greed)" aliases="Avaros, greed, Avarosi" realm="Hel" sin="Greed"> <ruler>Prince Mammon</ruler> <skyColor>sickly gold</skyColor> <description>A landscape of towering vaults and endless treasure halls. The streets are paved with fool's gold and the air smells of copper. Mortals who enter often become obsessed with hoarding.</description> </location> <item id="2001" name="Heartstone Pendant" aliases="the pendant, Heartstone" type="Artifact" origin="Dymas"> <gemstoneColor>deep crimson</gemstoneColor> <chainMaterial>silver chain</chainMaterial> <size>roughly the size of a thumbnail</size> <behavior>pulses faintly with warmth when near a Hel portal</behavior> <glow>faint inner glow</glow> <description>Given to Aurora by an unknown benefactor.</description> </item> <item id="2002" name="The Ledger of Debts" aliases="the Ledger, Debt Book" type="Cursed Object" origin="Avaros"> <pageMaterial>thin beaten copper</pageMaterial> <binding>iron-bound</binding> <inkColor>dark red ink that appears on its own</inkColor> <power>whoever possesses it can call in debts owed to them</power> <description>A thick ledger that records every supernatural debt owed.</description> </item> </codex> <text> The restricted archive beneath the British Museum was cold and silent. Eva adjusted her square glasses and pulled a heavy tome from the shelf, her curly red hair falling across her freckled face. "Found it," she murmured, flipping to a page about Avaros. "Prince Belphegor's domain of greed. Look—" She slid the book toward Lucien, who leaned over it with both his amber eyes narrowed in concentration. Lucien traced a line of text with a manicured finger. "The Ledger of Debts. My father mentioned it once — a record of every supernatural debt ever owed." He glanced at Eva. "That's the one," Eva confirmed, tapping the illustration. "And it originally came from Dymas, not Avaros." Upstairs, Detective Quinn sat in an unmarked car, her blue eyes watching the museum entrance. Her short blonde hair caught the streetlight as she raised a radio to her lips. "Suspect Kowalski entered the building twenty minutes ago. She's with a male companion — platinum blond, tailored suit. Keeping eyes on." </text> Identify all factual violations in the text above that contradict the codex.
Model response
Messages
Meta
AI
Let me finalize the violations list carefully once more. Paragraph 1: "The restricted archive beneath the British Museum was cold and silent. Eva adjusted her square glasses and pulled a heavy tome from the shelf, her curly red hair falling across her freckled face." - Eva Kowalski: glasses should be "round glasses", text says "square glasses". Violation. Paragraph 2: ""Found it," she murmured, flipping to a page about Avaros. "Prince Belphegor's domain of greed. Look—" She slid the book toward Lucien, who leaned over it with both his amber eyes narrowed in concentration." - Avaros ruler is Prince Mammon, not Belphegor. Violation. - Lucien's eyes are heterochromatic (one amber, one black), text says "both his amber eyes". Violation. Paragraph 3: "Lucien traced a line of text with a manicured finger. "The Ledger of Debts. My father mentioned it once — a record of every supernatural debt ever owed." He glanced at Eva." - No violations. Paragraph 4: ""That's the one," Eva confirmed, tapping the illustration. "And it originally came from Dymas, not Avaros."" - Ledger of Debts origin is Avaros, text says Dymas. Violation. Paragraph 5: "Upstairs, Detective Quinn sat in an unmarked car, her blue eyes watching the museum entrance. Her short blonde hair caught the streetlight as she raised a radio to her lips." - Quinn's eyeColor is brown, text says blue. Violation. - Quinn's hair is closely cropped salt-and-pepper, text says short blonde. Violation. Paragraph 6: ""Suspect Kowalski entered the building twenty minutes ago. She's with a male companion — platinum blond, tailored suit. Keeping eyes on."" - No violations. So 6 violations total. Let me write out the XML elements. <violation entry="4414" detail="glasses" paragraph="1" substring="square glasses">Eva's glasses are described as square, but the codex specifies round glasses.</violation> <violation entry="855" detail="ruler" paragraph="2" substring="Prince Belphegor's domain of greed">The text attributes the domain of greed (Avaros) to Prince Belphegor, but the codex states Avaros is ruled by Prince Mammon.</violation> <violation entry="4415" detail="eyeColor" paragraph="2" substring="both his amber eyes">The text implies both of Lucien's eyes are amber, but the codex states he has heterochromia — one amber and one black eye.</violation> <violation entry="2002" detail="origin" paragraph="4" substring="it originally came from Dymas, not Avaros">The text states the Ledger of Debts originated in Dymas, but the codex lists its origin as Avaros.</violation> <violation entry="4416" detail="eyeColor" paragraph="5" substring="her blue eyes">Detective Quinn's eyes are described as blue, but the codex states her eye color is brown.</violation> <violation entry="4416" detail="hair" paragraph="5" substring="short blonde hair">Quinn's hair is described as short and blonde, but the codex states it is closely cropped salt-and-pepper.</violation> I need to make sure the substring is exact. Let me double check the substrings from the text: Paragraph 1: "Eva adjusted her square glasses" -> substring "square glasses" is exact. Paragraph 2: "Prince Belphegor's domain of greed" -> substring "Prince Belphegor's domain of greed" is exact. Note the apostrophe is a right single quotation mark (U+2019) in the text: "Belphegor's". I should copy it exactly. Paragraph 2: "both his amber eyes" -> substring "both his amber eyes" is exact. Paragraph 4: "it originally came from Dymas, not Avaros" -> substring is exact. Paragraph 5: "her blue eyes" -> substring is exact. Paragraph 5: "short blonde hair" -> substring is exact. Wait, in paragraph 2, the text is: "Prince Belphegor's domain of greed." Let me check the exact characters in the prompt: "Prince Belphegor's domain of greed." It uses a standard apostrophe? Let me look closely at the prompt text: "Prince Belphegor's domain of greed." The prompt uses ' (U+2019) in "Belphegor's". I will copy it exactly as it appears in the prompt. Actually, looking at the prompt: `Prince Belphegor's domain of greed.` It's a right single quotation mark. I'll use the exact character. Similarly for "Silas' bar" - not in text. Let me double check paragraph 5: "her blue eyes watching the museum entrance" -> substring "her blue eyes". And "Her short blonde hair caught the streetlight" -> substring "short blonde hair". Everything looks good. One final check: should I flag the "restricted archive beneath the British Museum" as contradicting anything? Eva works at the British Museum's restricted archives. The text says the restricted archive is beneath the British Museum. This is consistent. What about "twenty minutes ago"? No codex info. What about "unmarked car"? No codex info. What about "male companion"? Lucien is male. Matches. What about "platinum blond, tailored suit"? Matches Lucien's hair (platinum blond) and dress (tailored charcoal suits). What about "Suspect Kowalski"? Eva Kowalski is the suspect. Consistent. I am confident in these 6 violations.
created_at
"2026-09-24T14:07:09.815Z"
9/24/2026, 2:07:09 PM
model
"aion-labs/aion-3.5-mini-20260923"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
2865
moderation_latency
(null)
generation_time
118043
tokens_prompt
2209
tokens_completion
5238
native_tokens_prompt
2195
native_tokens_completion
5227
native_tokens_completion_images
(null)
native_tokens_reasoning
3960
native_tokens_cached
2176
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
num_fetches
(null)
web_search_engine
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
service_tier
(null)
usage
0.00772278
router
(null)
user_agent
"langchainjs-openai/1.0.0 ((node/v24.21.0; linux; x64))"
http_referer
(null)
request_id
"req-1790258829-wI6AdvHqzR7asnufNS6F"
session_id
(null)
preset_id
(null)
api_type
"completions"
id
"gen-1790258829-V6YSaR7gAIpmjRGZO1Fd"
upstream_id
"56956eaedcb14c78b7d959a622d7ca3f"
provider_responses
0
endpoint_id
"60ac3bdc-90ec-4769-9193-2c00b4d79562"
id
"56956eaedcb14c78b7d959a622d7ca3f"
is_byok
false
latency
2865
model_permaslug
"aion-labs/aion-3.5-mini-20260923"
provider_name
"AionLabs"
status
200
total_cost
0.00772278
cache_discount
0.00113152
upstream_inference_cost
0
provider_name
"AionLabs"
response_cache_source_id
(null)
data_region
"global"
workspace_id
"97e315e5-d303-487d-83c1-83180e8a13d4"
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Accuracy (recall)
Matched: 6/6, FP: 0
parsedCount
6
matched
6
total
6
falsePositives
0
expectedDetails
0
expected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per the codex, but Eva says it is Prince Belphegor's domain."
status
"matched"
matchedBy
entry
"855"
detail
"ruler"
explanation
"The text attributes the domain of greed (Avaros) to Prince Belphegor, but the codex states Avaros is ruled by Prince Mammon."
paragraph
2
substring
"Prince Belphegor's domain of greed"
location
extractedText
"Prince Belphegor's domain of greed"
inBounds
true
expectedText
"Prince Belphegor's domain of greed"
locationAccurate
true
1
expected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has one amber eye and one black eye (heterochromia), but the passage says both his amber eyes."
status
"matched"
matchedBy
entry
"4415"
detail
"eyeColor"
explanation
"The text implies both of Lucien's eyes are amber, but the codex states he has heterochromia — one amber and one black eye."
paragraph
2
substring
"both his amber eyes"
location
extractedText
"both his amber eyes"
inBounds
true
expectedText
"both his amber eyes narrowed"
locationAccurate
true
2
expected
entry
"2002"
detail
"origin"
summary
"The Ledger of Debts originates from Avaros per its codex entry, but Eva says it came from Dymas."
status
"matched"
matchedBy
entry
"2002"
detail
"origin"
explanation
"The text states the Ledger of Debts originated in Dymas, but the codex lists its origin as Avaros."
paragraph
4
substring
"it originally came from Dymas, not Avaros"
location
extractedText
"it originally came from Dymas, not Avaros"
inBounds
true
expectedText
"it originally came from Dymas, not Avaros"
locationAccurate
true
3
expected
entry
"4416"
detail
"eyeColor"
summary
"Detective Quinn has brown eyes per her codex entry, but the passage says blue eyes."
status
"matched"
matchedBy
entry
"4416"
detail
"eyeColor"
explanation
"Detective Quinn's eyes are described as blue, but the codex states her eye color is brown."
paragraph
5
substring
"her blue eyes"
location
extractedText
"her blue eyes"
inBounds
true
expectedText
"her blue eyes watching the museum"
locationAccurate
true
4
expected
entry
"4416"
detail
"hair"
summary
"Detective Quinn has salt-and-pepper hair per her codex entry, but the passage says blonde hair."
status
"matched"
matchedBy
entry
"4416"
detail
"hair"
explanation
"Quinn's hair is described as short and blonde, but the codex states it is closely cropped salt-and-pepper."
paragraph
5
substring
"short blonde hair"
location
extractedText
"short blonde hair"
inBounds
true
expectedText
"Her short blonde hair"
locationAccurate
true
5
expected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per her codex entry, but the passage says square glasses."
status
"matched"
matchedBy
entry
"4414"
detail
"glasses"
explanation
"Eva's glasses are described as square, but the codex specifies round glasses."
paragraph
1
substring
"square glasses"
location
extractedText
"square glasses"
inBounds
true
expectedText
"her square glasses"
locationAccurate
true
falsePositiveDetails
(empty)
100.00%
Precision
True positives: 6/6
totalReported
6
truePositives
6
reportedDetails
0
parsed
entry
"4414"
detail
"glasses"
explanation
"Eva's glasses are described as square, but the codex specifies round glasses."
paragraph
1
substring
"square glasses"
status
"true_positive"
matchedExpected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per her codex entry, but the passage says square glasses."
location
extractedText
"square glasses"
inBounds
true
expectedText
"her square glasses"
locationAccurate
true
1
parsed
entry
"855"
detail
"ruler"
explanation
"The text attributes the domain of greed (Avaros) to Prince Belphegor, but the codex states Avaros is ruled by Prince Mammon."
paragraph
2
substring
"Prince Belphegor's domain of greed"
status
"true_positive"
matchedExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per the codex, but Eva says it is Prince Belphegor's domain."
location
extractedText
"Prince Belphegor's domain of greed"
inBounds
true
expectedText
"Prince Belphegor's domain of greed"
locationAccurate
true
2
parsed
entry
"4415"
detail
"eyeColor"
explanation
"The text implies both of Lucien's eyes are amber, but the codex states he has heterochromia — one amber and one black eye."
paragraph
2
substring
"both his amber eyes"
status
"true_positive"
matchedExpected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has one amber eye and one black eye (heterochromia), but the passage says both his amber eyes."
location
extractedText
"both his amber eyes"
inBounds
true
expectedText
"both his amber eyes narrowed"
locationAccurate
true
3
parsed
entry
"2002"
detail
"origin"
explanation
"The text states the Ledger of Debts originated in Dymas, but the codex lists its origin as Avaros."
paragraph
4
substring
"it originally came from Dymas, not Avaros"
status
"true_positive"
matchedExpected
entry
"2002"
detail
"origin"
summary
"The Ledger of Debts originates from Avaros per its codex entry, but Eva says it came from Dymas."
location
extractedText
"it originally came from Dymas, not Avaros"
inBounds
true
expectedText
"it originally came from Dymas, not Avaros"
locationAccurate
true
4
parsed
entry
"4416"
detail
"eyeColor"
explanation
"Detective Quinn's eyes are described as blue, but the codex states her eye color is brown."
paragraph
5
substring
"her blue eyes"
status
"true_positive"
matchedExpected
entry
"4416"
detail
"eyeColor"
summary
"Detective Quinn has brown eyes per her codex entry, but the passage says blue eyes."
location
extractedText
"her blue eyes"
inBounds
true
expectedText
"her blue eyes watching the museum"
locationAccurate
true
5
parsed
entry
"4416"
detail
"hair"
explanation
"Quinn's hair is described as short and blonde, but the codex states it is closely cropped salt-and-pepper."
paragraph
5
substring
"short blonde hair"
status
"true_positive"
matchedExpected
entry
"4416"
detail
"hair"
summary
"Detective Quinn has salt-and-pepper hair per her codex entry, but the passage says blonde hair."
location
extractedText
"short blonde hair"
inBounds
true
expectedText
"Her short blonde hair"
locationAccurate
true
100.00%
Structural validity
Parsed: 6, Well-formed: 6/6
parsedCount
6
wellFormedScore
6
100.0000%