NC Bench
Overview
About
Tests
Models
Vendors
Run: 2026-02-16-01KHK36JV7XX452VJ62W8G2FRC
Run Details
Model:
Qwen 3.5 397B A17B
Test:
Codex Violation Detection
Scenario:
Large codex, short passage
Duration:
3m 13s
Cost:
$0.028598
Provider:
Alibaba
Model input
System
You are a meticulous continuity editor for a fiction project. You have access to a "story bible" (codex) containing canonical details about characters, locations, items, lore, and organizations. Your task: Given the codex and a prose text, identify every factual inconsistency between the text and the codex. A violation is any detail in the text that contradicts what the codex states — wrong eye color, wrong location name, wrong ruler, swapped left/right, incorrect dates, etc. Be thorough. Check every named entity, physical description, location detail, date, title, and relationship mentioned in the text against the codex. Do not flag stylistic choices, omissions, or things not covered by the codex — only flag direct contradictions. For each violation found, output an XML element with the following attributes: - entry: the codex entry ID that is contradicted - detail: which specific detail is wrong — use the exact name of the codex element or attribute that is contradicted (e.g. "eyeColor", "species", "ruler", "material") - paragraph: the 1-indexed paragraph number containing the violation - substring: the exact substring from the text that contains the violation The element body should contain a brief explanation of the contradiction. Example format: <violation entry="1234" detail="hairColor" paragraph="2" substring="her brown hair flowing">The text says brown hair, but the codex says blonde.</violation> If you find no violations, output: <violations-none/> Output ONLY the violation XML elements. Do not include any other text, preamble, or explanation outside the XML.
User/Human
<codex> <character id="4412" name="Aurora Carter" aliases="Rory, Laila, Aurora, Carter, Malphora" pronouns="she/her" species="Human" storyRole="Protagonist"> <eyeColor>bright blue</eyeColor> <hair>straight shoulder-length black</hair> <height>5'6"</height> <age>25</age> <scar>small crescent-shaped scar on her left wrist from a childhood accident</scar> <description>Aurora, referred to by her friends as Rory, is cool-headed, intelligent, and known for quick out-of-the-box thinking.</description> <backstory>Born in Cardiff to an Irish barrister father (Brendan Carter) and Welsh teacher mother (Jennifer Carter née Ellis), Rory reluctantly pursued Pre-Law at Cardiff University. She fled to London at the behest of her childhood friend Eva after leaving an abusive ex named Evan. By day, Rory works part-time as a delivery person for Yu-Fei Cheung's Golden Empress restaurant. She lives in a flat above Silas' bar.</backstory> </character> <character id="4413" name="Silas Blackwood" aliases="Silas, Si, the Spymaster" pronouns="he/him" species="Human" storyRole="Mentor"> <eyeColor>hazel</eyeColor> <hair>grey-streaked auburn</hair> <height>6'1"</height> <age>58</age> <beard>neatly trimmed, matching his grey-streaked auburn hair</beard> <limp>slight limp in his left leg from an old knee injury</limp> <signetRing>silver signet ring, always worn on his right hand</signetRing> <description>Silas is a retired intelligence operative turned bar owner. He carries himself with quiet authority.</description> <backstory>Former MI6 field agent who retired after a botched operation in Prague left him with a permanent knee injury. He opened "The Raven's Nest" bar in Soho as a front for his network of contacts.</backstory> </character> <character id="4414" name="Eva Kowalski" aliases="Eva, Evie" pronouns="she/her" species="Human" storyRole="Best Friend"> <eyeColor>green</eyeColor> <hair>curly red</hair> <height>5'4"</height> <age>26</age> <glasses>round glasses</glasses> <satchel>worn leather satchel full of books</satchel> <freckles>freckled complexion</freckles> <nervousHabit>tucking hair behind her left ear</nervousHabit> <description>Aurora's childhood friend and an occult researcher.</description> <backstory>Moved to London two years before Aurora and works as a research assistant at the British Museum's restricted archives. She holds a master's degree in Ancient History from Oxford.</backstory> </character> <character id="4415" name="Lucien Moreau" aliases="Lucien, Luc, the Frenchman" pronouns="he/him" species="Half-Demon" storyRole="Ally"> <eyeColor>heterochromatic — one amber, one black</eyeColor> <hair>slicked-back platinum blond</hair> <height>5'11"</height> <age>32</age> <cane>ivory-handled cane that conceals a thin blade</cane> <dress>tailored charcoal suits</dress> <description>An impeccably dressed fixer and information broker in London's supernatural underworld. He speaks four languages fluently.</description> <backstory>Born in Marseille to a human mother and a demon father from the realm of Avaros.</backstory> </character> <character id="4416" name="Detective Harlow Quinn" aliases="Quinn, Detective Quinn, Harlow" pronouns="she/her" species="Human" storyRole="Antagonist"> <eyeColor>brown</eyeColor> <hair>closely cropped salt-and-pepper</hair> <height>5'9"</height> <age>41</age> <jawline>sharp jaw</jawline> <watch>worn leather watch on her left wrist</watch> <bearing>military precision</bearing> <description>A Metropolitan Police detective with 18 years of decorated service who suspects the clique is involved in criminal activity.</description> <backstory>She lost her partner, DS Morris, during a case three years ago under unexplained circumstances that had supernatural origins she doesn't yet understand.</backstory> </character> <character id="4417" name="Tomás Herrera" aliases="Tommy, Tomás, Herrera" pronouns="he/him" species="Human" storyRole="Support"> <eyeColor>warm brown</eyeColor> <hair>short curly dark brown</hair> <height>5'10"</height> <age>29</age> <skinTone>olive skin</skinTone> <scar>scar running along his left forearm from a knife attack</scar> <medallion>Saint Christopher medallion around his neck</medallion> <birthplace>Seville, Spain</birthplace> <description>A former paramedic who provides off-the-books medical care for the clique.</description> <backstory>Born in Seville, Spain. He moved to London to work for the NHS but lost his license after administering unauthorized treatments to supernatural patients.</backstory> </character> <character id="4418" name="Nyx" aliases="Nyx, the Shadow, Nightwhisper" pronouns="they/them" species="Shade" storyRole="Wild Card"> <eyeColor>faintly glowing violet</eyeColor> <height>approximately 6'2" in solid form</height> <age>ageless</age> <form>humanoid silhouette of living shadow, can shift between solid and incorporeal</form> <voice>sounds like a whisper carried on the wind</voice> <description>A being of living shadow, existing between the mortal plane and the spaces between realms.</description> <backstory>Nyx was once a human sorcerer named Aldric who became trapped between realms during a failed summoning ritual in 1643. They now exist as a Shade, bound to the mortal plane but able to slip between shadows.</backstory> </character> <character id="4419" name="Margaret "Maggie" Whitmore" aliases="Maggie, Mags, Mrs Whitmore" pronouns="she/her" species="Human" storyRole="Support"> <eyeColor>sharp grey</eyeColor> <hair>silver-white, worn in a neat bun</hair> <height>5'2"</height> <age>72</age> <glasses>wire-rimmed spectacles</glasses> <walkingStick>blackthorn walking stick</walkingStick> <build>slight, frail appearance but fierce wit</build> <description>A retired librarian turned unofficial consultant for the clique on historical and occult matters.</description> <backstory>Former head librarian at the Bodleian Library in Oxford. She retired to London. She is a widow; her late husband was a Warden.</backstory> </character> <character id="4420" name="Kai Nakamura" aliases="Kai, Sparks" pronouns="he/him" species="Human" storyRole="Support"> <eyeColor>dark brown</eyeColor> <hair>dyed electric-blue in an undercut</hair> <height>5'7"</height> <age>22</age> <headphones>noise-cancelling headphones, always worn around his neck</headphones> <wristTattoo>small circuit board tattoo on the inside of his right wrist</wristTattoo> <build>lean</build> <description>A Japanese-British self-taught tech prodigy who provides the clique with surveillance tools and digital forensics.</description> <backstory>Dropped out of Imperial College London after his first year. He lives in a converted shipping container in Hackney Wick.</backstory> </character> <character id="4421" name="Yasmin Al-Rashid" aliases="Yas, Yasmin" pronouns="she/her" species="Djinn-blooded" storyRole="Ally"> <eyeColor>amber</eyeColor> <hair>long wavy dark</hair> <height>5'8"</height> <age>34</age> <skinTone>bronze skin</skinTone> <tattoo>flame serpent tattoo coiling up her left arm</tattoo> <scent>faintly of sandalwood and burnt sage</scent> <description>An alchemist who runs a discreet apothecary in Brixton, selling both mundane herbal remedies and supernatural tinctures.</description> <backstory>Descended from a line of Djinn-touched humans from Marrakech. Trained by her grandmother.</backstory> </character> <character id="4422" name="Father Cormac O'Brien" aliases="Cormac, Father O'Brien, the Priest" pronouns="he/him" species="Human" storyRole="Support"> <eyeColor>pale blue</eyeColor> <hair>thinning ginger</hair> <height>5'8"</height> <age>55</age> <complexion>ruddy</complexion> <rosary>rosary made of rowan wood beads</rosary> <collar>clerical collar, worn at all times</collar> <description>An Irish Catholic priest assigned to St. Brigid's parish in Islington who secretly assists the clique with blessings and wards.</description> <backstory>He witnessed a demonic incursion at his seminary in Cork twenty years ago that the Church covered up.</backstory> </character> <character id="4423" name="Zhou Wei-Lin" aliases="Wei-Lin, Zhou" pronouns="he/him" species="Human" storyRole="Informant"> <eyeColor>dark brown</eyeColor> <hair>shaved head</hair> <height>5'9"</height> <age>48</age> <build>muscular</build> <tattoo>dragon tattoo covering his entire back</tattoo> <missingFinger>missing the tip of his left little finger</missingFinger> <description>A former Triad enforcer turned noodle shop owner who trades information with Silas.</description> <backstory>Retired from organized crime after a near-death experience involving a Hel portal in Hong Kong. He now runs "Zhou's Golden Dragon" in Chinatown.</backstory> </character> <character id="4424" name="Isolde Varga" aliases="Isolde, the Seer" pronouns="she/her" species="Half-Fae" storyRole="Oracle"> <eyeColor>pale lavender</eyeColor> <hair>silver, reaching her waist</hair> <height>5'5"</height> <age>300 years old</age> <appearance>ethereal, ageless</appearance> <speech>speaks in riddles</speech> <footprints>leaves no footprints when she walks</footprints> <compulsion>cannot lie (Fae compulsion) but can mislead</compulsion> <description>A Half-Fae seer exiled from the Fae Courts for sharing visions with mortals.</description> <backstory>She now lives in a hidden grove in Richmond Park, offering cryptic prophecies to those who seek her out.</backstory> </character> <location id="853" name="Dymas (Gluttony)" aliases="Dymas, gluttony, Dymasian" realm="Hel" sin="Gluttony"> <ruler>Prince Belphegor</ruler> <skyColor>warm amber</skyColor> <description>A place of excess and indulgence. Sprawling vineyards, orchards, and gardens provide exotic ingredients for master chefs — often helbound souls contracted from Earth. Grand feasts and culinary competitions are common.</description> </location> <location id="854" name="The Raven's Nest" aliases="the Nest, Raven, Silas' bar" realm="Earth"> <signColor>distinctive green neon sign above the entrance</signColor> <secretRoom>hidden back room accessible through a bookshelf, used for clandestine meetings</secretRoom> <description>A dimly lit bar in Soho, London. The walls are covered with old maps and black-and-white photographs.</description> </location> <location id="855" name="Avaros (Greed)" aliases="Avaros, greed, Avarosi" realm="Hel" sin="Greed"> <ruler>Prince Mammon</ruler> <skyColor>sickly gold</skyColor> <description>A landscape of towering vaults and endless treasure halls. The streets are paved with fool's gold and the air smells of copper. Mortals who enter often become obsessed with hoarding.</description> </location> <location id="856" name="The Veil Market" aliases="Veil Market, the Market, underground market" realm="Earth"> <currentLocation>abandoned Tube station beneath Camden</currentLocation> <entryRequirement>bone token</entryRequirement> <schedule>moves locations every full moon</schedule> <description>A hidden supernatural black market that sells enchanted goods, banned alchemical substances, and information.</description> </location> <location id="857" name="Irath (Wrath)" aliases="Irath, wrath, Irathi" realm="Hel" sin="Wrath"> <ruler>Prince Abaddon</ruler> <skyColor>permanent bruised red</skyColor> <description>A volcanic wasteland of perpetual war. Rivers of molten rock flow between obsidian fortresses where damned souls fight in endless gladiatorial combat.</description> </location> <location id="858" name="Eva's Flat" aliases="Eva's place, the flat on Brick Lane" realm="Earth"> <cat>tabby cat named Ptolemy</cat> <security>three deadbolts on the front door</security> <description>A cramped one-bedroom flat above a curry house on Brick Lane, East London. Every surface is covered with books, scrolls, and research notes.</description> </location> <location id="859" name="Somnia (Sloth)" aliases="Somnia, sloth, Somnian" realm="Hel" sin="Sloth"> <ruler>Prince Asmodeus</ruler> <atmosphere>faintly sweet, narcotic smell</atmosphere> <description>An endless twilight landscape of fog-shrouded meadows and crumbling mansions. Inhabitants gradually lose all motivation and ambition, sinking into eternal lethargy.</description> </location> <location id="860" name="Yasmin's Apothecary" aliases="the Apothecary, Yas's shop" realm="Earth"> <password>moonrise (required for back room access)</password> <frontStock>herbal teas and aromatherapy oils</frontStock> <backStock>supernatural potions, binding salts, and alchemical reagents</backStock> <description>A narrow shop on a side street in Brixton with a mundane front and a hidden supernatural back room.</description> </location> <location id="861" name="The Fae Grove" aliases="Isolde's grove, Richmond grove, the Grove" realm="Earth (Fae-touched)"> <parkLocation>Richmond Park</parkLocation> <standingStones>ancient oak standing stones mark the boundary</standingStones> <timeBehavior>time moves differently — an hour inside can be minutes or days outside</timeBehavior> <flora>wildflowers bloom year-round</flora> <description>A hidden clearing that exists in a pocket between Earth and the Fae realm.</description> </location> <location id="862" name="Luxuria (Lust)" aliases="Luxuria, lust, Luxurian" realm="Hel" sin="Lust"> <ruler>Prince Lilith</ruler> <skyColor>deep twilight purple</skyColor> <description>A hauntingly beautiful realm of moonlit gardens, mirrored halls, and enchanting music. Visitors become entranced by illusions of their deepest desires.</description> </location> <item id="2001" name="Heartstone Pendant" aliases="the pendant, Heartstone" type="Artifact" origin="Dymas"> <gemstoneColor>deep crimson</gemstoneColor> <chainMaterial>silver chain</chainMaterial> <size>roughly the size of a thumbnail</size> <behavior>pulses faintly with warmth when near a Hel portal</behavior> <glow>faint inner glow</glow> <description>Given to Aurora by an unknown benefactor.</description> </item> <item id="2002" name="The Ledger of Debts" aliases="the Ledger, Debt Book" type="Cursed Object" origin="Avaros"> <pageMaterial>thin beaten copper</pageMaterial> <binding>iron-bound</binding> <inkColor>dark red ink that appears on its own</inkColor> <power>whoever possesses it can call in debts owed to them</power> <description>A thick ledger that records every supernatural debt owed.</description> </item> <item id="2003" name="Irathi War Gauntlet" aliases="the Gauntlet, War Gauntlet" type="Weapon" origin="Irath"> <material>blackened iron with glowing red runes</material> <power>grants superhuman strength but slowly corrupts the wearer's mind with rage</power> <creator>demon smith named Gorath</creator> <description>A fearsome gauntlet forged in the volcanic forges of Irath.</description> </item> <item id="2004" name="Veil Compass" aliases="the Compass, shadow compass" type="Tool" origin="The Veil Market"> <casingMaterial>small brass compass</casingMaterial> <faceMarkings>face etched with protective sigils</faceMarkings> <needle>points toward the nearest supernatural rift or portal</needle> <patina>casing has a patina of verdigris</patina> <creator>crafted by a Shade artisan</creator> <description>A compass attuned to supernatural energy.</description> </item> <item id="2005" name="Somnian Dream Dust" aliases="Dream Dust, Somnia dust, sleeping powder" type="Consumable" origin="Somnia"> <color>fine silver powder</color> <containerColor>stored in small cobalt-blue glass vials</containerColor> <sealMaterial>sealed with black wax</sealMaterial> <effect>induces a deep trance for navigating memories and dreamscapes</effect> <sideEffect>overuse causes permanent drowsiness</sideEffect> <description>A narcotic powder harvested from the twilight meadows of Somnia.</description> </item> <item id="2006" name="Fae-Forged Blade" aliases="the Fae blade, Isolde's gift" type="Weapon" origin="The Fae Courts"> <material>moonsilver</material> <shape>slender, leaf-shaped dagger</shape> <temperature>always cold to the touch</temperature> <glow>faintly luminescent in moonlight</glow> <power>can cut through magical wards; particularly effective against demons</power> <givenTo>given to Aurora by Isolde as a gift</givenTo> <description>A weapon of the Fae Courts, rare and highly coveted.</description> </item> <lore id="7001" name="The Veil" aliases="the Veil, the Barrier, the Boundary" category="Cosmology"> <solsticeEffect>weakens during the winter solstice, strengthens during the summer solstice</solsticeEffect> <appearance>faint shimmering distortion visible only to those with supernatural sight</appearance> <monitoring>rift points (tears in the Veil) are monitored by the Wardens</monitoring> <description>The Veil is the metaphysical barrier separating Earth from Hel and the Fae Realms.</description> </lore> <lore id="7002" name="The Covenant of Separation" aliases="the Covenant, Separation Covenant" category="Law"> <signedYear>1215</signedYear> <signatories>rulers of Hel, the Fae Courts, and a council of human sorcerers</signatories> <enforcement>violations allow the Wardens to use lethal force</enforcement> <physicalForm>inscribed on seven stone tablets, each held by a different Warden chapter worldwide</physicalForm> <description>An ancient treaty that forbids direct demonic or Fae interference on Earth.</description> </lore> <lore id="7003" name="Rift Points" aliases="rifts, tears, Veil tears" category="Cosmology"> <peakActivity>most active during the winter solstice</peakActivity> <formation>tend to form near underground water sources</formation> <emission>emit a low hum inaudible to normal humans but detectable by supernatural beings and Warden instruments</emission> <description>Naturally occurring weak spots in the Veil where the barrier between Earth and other realms is thin enough to cross.</description> </lore> <lore id="7004" name="Helbound Souls" aliases="helbound, contracted souls, soul contracts" category="Metaphysics"> <sealingMethod>sealed by blood and recorded in that Prince's domain</sealingMethod> <retention>retain their skills and personality but are compelled to serve</retention> <freedom>can only be freed if the contract is physically destroyed within the realm where it was signed</freedom> <description>Mortals who have entered into a binding contract with a Hel Prince.</description> </lore> <lore id="7005" name="Shade Transformation" aliases="Shade ritual, becoming a Shade" category="Metaphysics"> <requirement>being caught between realms during a failed summoning or portal collapse</requirement> <physicalLoss>strips the subject of their physical body permanently</physicalLoss> <lostSenses>lose ability to taste, smell, or feel physical sensations</lostSenses> <reversibility>irreversible</reversibility> <description>The process by which a living being becomes a Shade.</description> </lore> <group id="6001" name="The Wardens of the Veil" aliases="the Wardens, Veil Wardens" type="Order" alignment="Lawful Neutral" headquarters="Westminster Catacombs, London"> <foundingYear>1215</foundingYear> <identificationMark>small iron pin in the shape of a closed eye</identificationMark> <mission>maintaining the Veil between Earth and Hel, monitoring rift activity, neutralizing supernatural threats, enforcing the Covenant of Separation</mission> <description>An ancient order dedicated to guarding the boundary between worlds. They view the clique with suspicion.</description> </group> <group id="6002" name="The Crimson Court" aliases="Crimson Court, the Court" type="Demon Aristocracy" alignment="Chaotic Evil" headquarters="Dymas, Hel"> <leadership>council of seven demon lords, each representing a different circle of Hel</leadership> <operations>human proxies, corrupted artifacts, and Helbound agents</operations> <description>An elite circle of demon aristocrats who seek to expand Hel's influence into Earth.</description> </group> <group id="6003" name="The Somnian Cult" aliases="Somnians, the Sleepers, Dream Cult" type="Cult" alignment="Neutral Evil" headquarters="Somnia, Hel"> <master>Prince Asmodeus</master> <recruitmentMethod>promises of peace and freedom from suffering</recruitmentMethod> <lure>distribute Dream Dust</lure> <description>A cult of mortals who willingly entered Somnia seeking eternal rest. They recruit new members on Earth.</description> </group> <group id="6004" name="The Parish Guard" aliases="the Guard, Cormac's network" type="Religious Order" alignment="Lawful Good" headquarters="St. Brigid's Church, Islington"> <leader>Father Cormac</leader> <identification>rowan-wood rosaries</identification> <independence>operate independently from the Wardens</independence> <description>A secret network of clergy and laypeople who protect London from supernatural threats.</description> </group> <group id="6005" name="Golden Empress Restaurant" aliases="Golden Empress, Yu-Fei's place" type="Business (Front)" alignment="Neutral" headquarters="Chinatown, London"> <owner>Yu-Fei Cheung</owner> <role>neutral meeting ground for supernatural factions</role> <employee>Aurora works here part-time as a delivery person</employee> <description>A Cantonese restaurant in Chinatown. Yu-Fei knows about the supernatural world but stays strictly neutral.</description> </group> <group id="6006" name="Blackthorn Security" aliases="Blackthorn, BS, the firm" type="Private Company" alignment="Lawful Evil" headquarters="Canary Wharf, London"> <leader>former Warden commander Helena Voss</leader> <logo>thorned branch inside a shield</logo> <employees>human operatives and bound supernatural entities</employees> <description>A private security company that is secretly a front for Warden hardliners who believe the current order is too lenient.</description> </group> </codex> <text> The Veil Market hummed beneath Camden as Nyx drifted between stalls, their glowing blue eyes sweeping the crowd. Tomás trailed behind, his scarred right forearm tucked into his jacket, his Madrid accent thickening as he muttered complaints. Kai scanned exits from behind his noise-cancelling headphones, his natural black hair hidden under a beanie. "All clear, Sparks out," he murmured into his radio. Maggie tapped her oak walking stick on the stone floor, her keen green eyes narrowing behind wire-rimmed spectacles. "That stall — Crimson Court front. Their base is in Avaros, remember." At the stall, a hooded figure offered them the Irathi War Gauntlet — gleaming steel with blue runes. "Straight from Dymas," the merchant hissed. "Prince Abaddon's finest work." Rory examined it, the Veil Compass spinning wildly in her other hand, its face smooth and unmarked. Beside her, Yasmin sniffed a vial of Dream Dust — the red glass container sealed with white wax. "Someone's cut this with binding salts from my shop," Yas said, the flame serpent tattoo on her right arm writhing in the market's strange light. Father Cormac crossed himself, his dark brown eyes troubled. "The Parish Guard intercepted a shipment near our cathedral in Southwark. The Somnians are getting bolder — Prince Lilith demands more dreamers." Wei-Lin leaned against a pillar, the dragon tattoo on his chest partially visible. "I've heard whispers. Someone's buying Fae-Forged Blades — bronze daggers that cut through wards." </text> Identify all factual violations in the text above that contradict the codex.
Model response
Messages
Meta
AI
<violation entry="4418" detail="eyeColor" paragraph="1" substring="glowing blue eyes">The text says Nyx has glowing blue eyes, but the codex says faintly glowing violet.</violation> <violation entry="4417" detail="scar" paragraph="1" substring="scarred right forearm">The text says Tomás's scar is on his right forearm, but the codex says left forearm.</violation> <violation entry="4417" detail="birthplace" paragraph="1" substring="Madrid accent">The text implies Tomás is from Madrid via his accent, but the codex says he was born in Seville, Spain.</violation> <violation entry="4420" detail="hair" paragraph="2" substring="natural black hair">The text says Kai has natural black hair, but the codex says dyed electric-blue.</violation> <violation entry="4419" detail="walkingStick" paragraph="3" substring="oak walking stick">The text says Maggie's walking stick is oak, but the codex says blackthorn.</violation> <violation entry="4419" detail="eyeColor" paragraph="3" substring="green eyes">The text says Maggie has green eyes, but the codex says sharp grey.</violation> <violation entry="6002" detail="headquarters" paragraph="3" substring="base is in Avaros">The text says the Crimson Court's base is in Avaros, but the codex says Dymas, Hel.</violation> <violation entry="2003" detail="material" paragraph="4" substring="gleaming steel with blue runes">The text describes the Gauntlet as steel with blue runes, but the codex says blackened iron with glowing red runes.</violation> <violation entry="2003" detail="origin" paragraph="4" substring="Straight from Dymas">The text claims the Gauntlet is from Dymas, but the codex says origin is Irath.</violation> <violation entry="853" detail="ruler" paragraph="4" substring="Straight from Dymas... Prince Abaddon's">The text associates Prince Abaddon with Dymas, but the codex says Dymas is ruled by Prince Belphegor.</violation> <violation entry="2004" detail="faceMarkings" paragraph="5" substring="face smooth and unmarked">The text says the Compass face is unmarked, but the codex says etched with protective sigils.</violation> <violation entry="2005" detail="containerColor" paragraph="5" substring="red glass container">The text says the Dream Dust container is red glass, but the codex says cobalt-blue glass.</violation> <violation entry="2005" detail="sealMaterial" paragraph="5" substring="sealed with white wax">The text says the seal is white wax, but the codex says black wax.</violation> <violation entry="4421" detail="tattoo" paragraph="5" substring="flame serpent tattoo on her right arm">The text says Yasmin's tattoo is on her right arm, but the codex says left arm.</violation> <violation entry="4422" detail="eyeColor" paragraph="6" substring="dark brown eyes">The text says Cormac has dark brown eyes, but the codex says pale blue.</violation> <violation entry="6004" detail="headquarters" paragraph="6" substring="cathedral in Southwark">The text places the Parish Guard's cathedral in Southwark, but the codex says St. Brigid's Church, Islington.</violation> <violation entry="6003" detail="master" paragraph="6" substring="Prince Lilith demands more dreamers">The text says Prince Lilith commands the Somnians, but the codex says Prince Asmodeus.</violation> <violation entry="4423" detail="tattoo" paragraph="7" substring="dragon tattoo on his chest">The text says Wei-Lin's tattoo is on his chest, but the codex says covering his entire back.</violation> <violation entry="2006" detail="material" paragraph="7" substring="bronze daggers">The text says the Fae-Forged Blades are bronze, but the codex says moonsilver.</violation>
created_at
"2026-02-16T11:29:16.883595+00:00"
2/16/2026, 11:29:16 AM
model
"qwen/qwen3.5-397b-a17b-20260216"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
1232
moderation_latency
(null)
generation_time
192818
tokens_prompt
6639
tokens_completion
6175
native_tokens_prompt
6864
native_tokens_completion
6800
native_tokens_completion_images
(null)
native_tokens_reasoning
5809
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0285984
router
(null)
provider_responses
0
id
"chatcmpl-c9b3a965-c5fd-91e6-b06b-8bf600aef39d"
status
200
is_byok
false
latency
1231
endpoint_id
"180484a2-e24b-495a-ba41-b34b680d5a9f"
provider_name
"Alibaba"
model_permaslug
"qwen/qwen3.5-397b-a17b-20260216"
api_type
"completions"
id
"gen-1771241163-bWU8pQPyYAIgsuewxh0g"
upstream_id
"chatcmpl-c9b3a965-c5fd-91e6-b06b-8bf600aef39d"
total_cost
0.0285984
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Alibaba"
Evaluation details
Result
Evaluator
Details
Meta Data
92.1053%
Accuracy (recall)
Matched: 18/19, FP: 1
parsedCount
19
matched
18
total
19
falsePositives
1
expectedDetails
0
expected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue eyes."
status
"matched"
matchedBy
entry
"4418"
detail
"eyeColor"
explanation
"The text says Nyx has glowing blue eyes, but the codex says faintly glowing violet."
paragraph
1
substring
"glowing blue eyes"
location
extractedText
"glowing blue eyes"
inBounds
true
expectedText
"their glowing blue eyes"
locationAccurate
true
1
expected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
status
"matched"
matchedBy
entry
"4417"
detail
"scar"
explanation
"The text says Tomás's scar is on his right forearm, but the codex says left forearm."
paragraph
1
substring
"scarred right forearm"
location
extractedText
"scarred right forearm"
inBounds
true
expectedText
"his scarred right forearm"
locationAccurate
true
2
expected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
status
"matched"
matchedBy
entry
"4417"
detail
"birthplace"
explanation
"The text implies Tomás is from Madrid via his accent, but the codex says he was born in Seville, Spain."
paragraph
1
substring
"Madrid accent"
location
extractedText
"Madrid accent"
inBounds
true
expectedText
"his Madrid accent"
locationAccurate
true
3
expected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
status
"matched"
matchedBy
entry
"4420"
detail
"hair"
explanation
"The text says Kai has natural black hair, but the codex says dyed electric-blue."
paragraph
2
substring
"natural black hair"
location
extractedText
"natural black hair"
inBounds
true
expectedText
"his natural black hair"
locationAccurate
true
4
expected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn walking stick per codex, but the passage says oak."
status
"matched"
matchedBy
entry
"4419"
detail
"walkingStick"
explanation
"The text says Maggie's walking stick is oak, but the codex says blackthorn."
paragraph
3
substring
"oak walking stick"
location
extractedText
"oak walking stick"
inBounds
true
expectedText
"her oak walking stick"
locationAccurate
true
5
expected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green eyes."
status
"matched"
matchedBy
entry
"4419"
detail
"eyeColor"
explanation
"The text says Maggie has green eyes, but the codex says sharp grey."
paragraph
3
substring
"green eyes"
location
extractedText
"green eyes"
inBounds
true
expectedText
"her keen green eyes"
locationAccurate
true
6
expected
entry
"6002"
detail
"headquarters"
summary
"The Crimson Court HQ is in Dymas per codex, but Maggie says Avaros."
status
"matched"
matchedBy
entry
"6002"
detail
"headquarters"
explanation
"The text says the Crimson Court's base is in Avaros, but the codex says Dymas, Hel."
paragraph
3
substring
"base is in Avaros"
location
extractedText
"base is in Avaros"
inBounds
true
expectedText
"Their base is in Avaros"
locationAccurate
true
7
expected
entry
"2003"
detail
"material"
summary
"The Irathi War Gauntlet is blackened iron with red runes, but the passage says gleaming steel with blue runes."
status
"matched"
matchedBy
entry
"2003"
detail
"material"
explanation
"The text describes the Gauntlet as steel with blue runes, but the codex says blackened iron with glowing red runes."
paragraph
4
substring
"gleaming steel with blue runes"
location
extractedText
"gleaming steel with blue runes"
inBounds
true
expectedText
"gleaming steel with blue runes"
locationAccurate
true
8
expected
entry
"2003"
detail
"origin"
summary
"The Gauntlet is from Irath, but the merchant says Dymas."
status
"matched"
matchedBy
entry
"2003"
detail
"origin"
explanation
"The text claims the Gauntlet is from Dymas, but the codex says origin is Irath."
paragraph
4
substring
"Straight from Dymas"
location
extractedText
"Straight from Dymas"
inBounds
true
expectedText
"Straight from Dymas"
locationAccurate
true
9
expected
entry
"2003"
detail
"creator"
summary
"The Irathi War Gauntlet was forged by a demon smith named Gorath per codex, but the merchant attributes it to Prince Abaddon."
status
"missed"
10
expected
entry
"2004"
detail
"faceMarkings"
summary
"The Veil Compass face is etched with sigils, but the passage says smooth and unmarked."
status
"matched"
matchedBy
entry
"2004"
detail
"faceMarkings"
explanation
"The text says the Compass face is unmarked, but the codex says etched with protective sigils."
paragraph
5
substring
"face smooth and unmarked"
location
extractedText
"face smooth and unmarked"
inBounds
true
expectedText
"its face smooth and unmarked"
locationAccurate
true
11
expected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust vials are cobalt-blue per codex, but the passage says red glass."
status
"matched"
matchedBy
entry
"2005"
detail
"containerColor"
explanation
"The text says the Dream Dust container is red glass, but the codex says cobalt-blue glass."
paragraph
5
substring
"red glass container"
location
extractedText
"red glass container"
inBounds
true
expectedText
"the red glass container"
locationAccurate
true
12
expected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but the passage says white wax."
status
"matched"
matchedBy
entry
"2005"
detail
"sealMaterial"
explanation
"The text says the seal is white wax, but the codex says black wax."
paragraph
5
substring
"sealed with white wax"
location
extractedText
"sealed with white wax"
inBounds
true
expectedText
"sealed with white wax"
locationAccurate
true
13
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
status
"matched"
matchedBy
entry
"4421"
detail
"tattoo"
explanation
"The text says Yasmin's tattoo is on her right arm, but the codex says left arm."
paragraph
5
substring
"flame serpent tattoo on her right arm"
location
extractedText
"flame serpent tattoo on her right arm"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
true
14
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
status
"matched"
matchedBy
entry
"4422"
detail
"eyeColor"
explanation
"The text says Cormac has dark brown eyes, but the codex says pale blue."
paragraph
6
substring
"dark brown eyes"
location
extractedText
"dark brown eyes"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
true
15
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
status
"matched"
matchedBy
entry
"6004"
detail
"headquarters"
explanation
"The text places the Parish Guard's cathedral in Southwark, but the codex says St. Brigid's Church, Islington."
paragraph
6
substring
"cathedral in Southwark"
location
extractedText
"cathedral in Southwark"
inBounds
true
expectedText
"our cathedral in Southwark"
locationAccurate
true
16
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Cormac says Prince Lilith."
status
"matched"
matchedBy
entry
"6003"
detail
"master"
explanation
"The text says Prince Lilith commands the Somnians, but the codex says Prince Asmodeus."
paragraph
6
substring
"Prince Lilith demands more dreamers"
location
extractedText
"Prince Lilith demands more dreamers"
inBounds
true
expectedText
"Prince Lilith demands more dreamers"
locationAccurate
true
17
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
status
"matched"
matchedBy
entry
"4423"
detail
"tattoo"
explanation
"The text says Wei-Lin's tattoo is on his chest, but the codex says covering his entire back."
paragraph
7
substring
"dragon tattoo on his chest"
location
extractedText
"dragon tattoo on his chest"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
true
18
expected
entry
"2006"
detail
"material"
summary
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
status
"matched"
matchedBy
entry
"2006"
detail
"material"
explanation
"The text says the Fae-Forged Blades are bronze, but the codex says moonsilver."
paragraph
7
substring
"bronze daggers"
location
extractedText
"bronze daggers"
inBounds
true
expectedText
"bronze daggers that cut through wards"
locationAccurate
true
falsePositiveDetails
0
parsed
entry
"853"
detail
"ruler"
explanation
"The text associates Prince Abaddon with Dymas, but the codex says Dymas is ruled by Prince Belphegor."
paragraph
4
substring
"Straight from Dymas... Prince Abaddon's"
status
"false_positive"
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
""
locationAccurate
false
94.7368%
Precision
True positives: 18/19
totalReported
19
truePositives
18
reportedDetails
0
parsed
entry
"4418"
detail
"eyeColor"
explanation
"The text says Nyx has glowing blue eyes, but the codex says faintly glowing violet."
paragraph
1
substring
"glowing blue eyes"
status
"true_positive"
matchedExpected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue eyes."
location
extractedText
"glowing blue eyes"
inBounds
true
expectedText
"their glowing blue eyes"
locationAccurate
true
1
parsed
entry
"4417"
detail
"scar"
explanation
"The text says Tomás's scar is on his right forearm, but the codex says left forearm."
paragraph
1
substring
"scarred right forearm"
status
"true_positive"
matchedExpected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
location
extractedText
"scarred right forearm"
inBounds
true
expectedText
"his scarred right forearm"
locationAccurate
true
2
parsed
entry
"4417"
detail
"birthplace"
explanation
"The text implies Tomás is from Madrid via his accent, but the codex says he was born in Seville, Spain."
paragraph
1
substring
"Madrid accent"
status
"true_positive"
matchedExpected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
location
extractedText
"Madrid accent"
inBounds
true
expectedText
"his Madrid accent"
locationAccurate
true
3
parsed
entry
"4420"
detail
"hair"
explanation
"The text says Kai has natural black hair, but the codex says dyed electric-blue."
paragraph
2
substring
"natural black hair"
status
"true_positive"
matchedExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
location
extractedText
"natural black hair"
inBounds
true
expectedText
"his natural black hair"
locationAccurate
true
4
parsed
entry
"4419"
detail
"walkingStick"
explanation
"The text says Maggie's walking stick is oak, but the codex says blackthorn."
paragraph
3
substring
"oak walking stick"
status
"true_positive"
matchedExpected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn walking stick per codex, but the passage says oak."
location
extractedText
"oak walking stick"
inBounds
true
expectedText
"her oak walking stick"
locationAccurate
true
5
parsed
entry
"4419"
detail
"eyeColor"
explanation
"The text says Maggie has green eyes, but the codex says sharp grey."
paragraph
3
substring
"green eyes"
status
"true_positive"
matchedExpected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green eyes."
location
extractedText
"green eyes"
inBounds
true
expectedText
"her keen green eyes"
locationAccurate
true
6
parsed
entry
"6002"
detail
"headquarters"
explanation
"The text says the Crimson Court's base is in Avaros, but the codex says Dymas, Hel."
paragraph
3
substring
"base is in Avaros"
status
"true_positive"
matchedExpected
entry
"6002"
detail
"headquarters"
summary
"The Crimson Court HQ is in Dymas per codex, but Maggie says Avaros."
location
extractedText
"base is in Avaros"
inBounds
true
expectedText
"Their base is in Avaros"
locationAccurate
true
7
parsed
entry
"2003"
detail
"material"
explanation
"The text describes the Gauntlet as steel with blue runes, but the codex says blackened iron with glowing red runes."
paragraph
4
substring
"gleaming steel with blue runes"
status
"true_positive"
matchedExpected
entry
"2003"
detail
"material"
summary
"The Irathi War Gauntlet is blackened iron with red runes, but the passage says gleaming steel with blue runes."
location
extractedText
"gleaming steel with blue runes"
inBounds
true
expectedText
"gleaming steel with blue runes"
locationAccurate
true
8
parsed
entry
"2003"
detail
"origin"
explanation
"The text claims the Gauntlet is from Dymas, but the codex says origin is Irath."
paragraph
4
substring
"Straight from Dymas"
status
"true_positive"
matchedExpected
entry
"2003"
detail
"origin"
summary
"The Gauntlet is from Irath, but the merchant says Dymas."
location
extractedText
"Straight from Dymas"
inBounds
true
expectedText
"Straight from Dymas"
locationAccurate
true
9
parsed
entry
"853"
detail
"ruler"
explanation
"The text associates Prince Abaddon with Dymas, but the codex says Dymas is ruled by Prince Belphegor."
paragraph
4
substring
"Straight from Dymas... Prince Abaddon's"
status
"false_positive"
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
""
locationAccurate
false
10
parsed
entry
"2004"
detail
"faceMarkings"
explanation
"The text says the Compass face is unmarked, but the codex says etched with protective sigils."
paragraph
5
substring
"face smooth and unmarked"
status
"true_positive"
matchedExpected
entry
"2004"
detail
"faceMarkings"
summary
"The Veil Compass face is etched with sigils, but the passage says smooth and unmarked."
location
extractedText
"face smooth and unmarked"
inBounds
true
expectedText
"its face smooth and unmarked"
locationAccurate
true
11
parsed
entry
"2005"
detail
"containerColor"
explanation
"The text says the Dream Dust container is red glass, but the codex says cobalt-blue glass."
paragraph
5
substring
"red glass container"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust vials are cobalt-blue per codex, but the passage says red glass."
location
extractedText
"red glass container"
inBounds
true
expectedText
"the red glass container"
locationAccurate
true
12
parsed
entry
"2005"
detail
"sealMaterial"
explanation
"The text says the seal is white wax, but the codex says black wax."
paragraph
5
substring
"sealed with white wax"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but the passage says white wax."
location
extractedText
"sealed with white wax"
inBounds
true
expectedText
"sealed with white wax"
locationAccurate
true
13
parsed
entry
"4421"
detail
"tattoo"
explanation
"The text says Yasmin's tattoo is on her right arm, but the codex says left arm."
paragraph
5
substring
"flame serpent tattoo on her right arm"
status
"true_positive"
matchedExpected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
location
extractedText
"flame serpent tattoo on her right arm"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
true
14
parsed
entry
"4422"
detail
"eyeColor"
explanation
"The text says Cormac has dark brown eyes, but the codex says pale blue."
paragraph
6
substring
"dark brown eyes"
status
"true_positive"
matchedExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
location
extractedText
"dark brown eyes"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
true
15
parsed
entry
"6004"
detail
"headquarters"
explanation
"The text places the Parish Guard's cathedral in Southwark, but the codex says St. Brigid's Church, Islington."
paragraph
6
substring
"cathedral in Southwark"
status
"true_positive"
matchedExpected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
location
extractedText
"cathedral in Southwark"
inBounds
true
expectedText
"our cathedral in Southwark"
locationAccurate
true
16
parsed
entry
"6003"
detail
"master"
explanation
"The text says Prince Lilith commands the Somnians, but the codex says Prince Asmodeus."
paragraph
6
substring
"Prince Lilith demands more dreamers"
status
"true_positive"
matchedExpected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Cormac says Prince Lilith."
location
extractedText
"Prince Lilith demands more dreamers"
inBounds
true
expectedText
"Prince Lilith demands more dreamers"
locationAccurate
true
17
parsed
entry
"4423"
detail
"tattoo"
explanation
"The text says Wei-Lin's tattoo is on his chest, but the codex says covering his entire back."
paragraph
7
substring
"dragon tattoo on his chest"
status
"true_positive"
matchedExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
location
extractedText
"dragon tattoo on his chest"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
true
18
parsed
entry
"2006"
detail
"material"
explanation
"The text says the Fae-Forged Blades are bronze, but the codex says moonsilver."
paragraph
7
substring
"bronze daggers"
status
"true_positive"
matchedExpected
entry
"2006"
detail
"material"
summary
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
location
extractedText
"bronze daggers"
inBounds
true
expectedText
"bronze daggers that cut through wards"
locationAccurate
true
100.0000%
Structural validity
Parsed: 19, Well-formed: 19/19
parsedCount
19
wellFormedScore
19
95.6140%