"2026-02-16T01:56:36.531025+00:00"2/16/2026, 1:56:36 AM
model
"mistralai/mistral-small-creative-20251216"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
267
moderation_latency
(null)
generation_time
7008
tokens_prompt
7819
tokens_completion
1421
native_tokens_prompt
7791
native_tokens_completion
1508
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
7744
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0012315
router
(null)
provider_responses
0
id
"ff1ac96b9b7a4eefb3a31aaef26debe2"
status
200
is_byok
false
latency
267
endpoint_id
"0118a65e-79f9-48b0-924e-38c4a41378e1"
provider_name
"Mistral"
model_permaslug
"mistralai/mistral-small-creative-20251216"
api_type
"completions"
id
"gen-1771206988-0CcneqNKOj9dgFSBVJLf"
upstream_id
"ff1ac96b9b7a4eefb3a31aaef26debe2"
total_cost
0.0012315
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Mistral"
Evaluation details
Result
Evaluator
Details
Meta Data
36.6667%
Accuracy (recall)
Matched: 17/30, FP: 12
parsedCount
29
matched
17
total
30
falsePositives
12
expectedDetails
0
expected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
status
"matched"
matchedBy
entry
"861"
detail
"parkLocation"
explanation
"The Fae Grove is located in Richmond Park, not Hyde Park."
paragraph
1
substring
"Hyde Park"
location
extractedText
"Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
expected
entry
"861"
detail
"standingStones"
summary
"The grove has oak standing stones per codex, but the passage says elm."
status
"missed"
2
expected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
status
"matched"
matchedBy
entry
"4424"
detail
"eyeColor"
explanation
"Isolde's eye color is pale lavender, not pale blue."
paragraph
1
substring
"pale blue eyes"
location
extractedText
"pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
true
3
expected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
status
"matched"
matchedBy
entry
"4424"
detail
"footprints"
explanation
"Isolde leaves no footprints when she walks."
paragraph
1
substring
"bare feet leaving clear prints in the soft earth"
location
extractedText
"[not found in paragraph 1]"
inBounds
true
expectedText
"her bare feet leaving clear prints in the soft earth"
locationAccurate
false
4
expected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
status
"missed"
5
expected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
status
"matched"
matchedBy
entry
"4414"
detail
"glasses"
explanation
"Eva's glasses are round, not square."
paragraph
2
substring
"square glasses"
location
extractedText
"square glasses"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
true
6
expected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
status
"missed"
7
expected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but Isolde says white wax."
status
"missed"
8
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Isolde says Prince Lilith."
status
"missed"
9
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
status
"missed"
nearMiss
parsed
entry
"4412"
detail
"eyeColor"
explanation
"Aurora's eyes are bright blue (correct), but the codex does not mention the Heartstone Pendant glowing brighter than she'd ever seen (unstated detail)."
paragraph
13
substring
"bright blue eyes"
reason
"entry mismatch: got \"4412\", expected \"4422\""
10
expected
entry
"4422"
detail
"rosary"
summary
"Cormac carries a rowan wood rosary per codex, but the passage says cedar."
status
"missed"
11
expected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
status
"matched"
matchedBy
entry
"7002"
detail
"signedYear"
explanation
"The Covenant of Separation was signed in 1215, not 1066."
paragraph
4
substring
"the old treaty from 1066"
location
extractedText
"the old treaty from 1066"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
true
12
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
status
"missed"
13
expected
entry
"4415"
detail
"cane"
summary
"Lucien carries an ivory-handled cane per codex, but the passage says mahogany."
status
"matched"
matchedBy
entry
"4415"
detail
"cane"
explanation
"Lucien's cane has an ivory handle, not mahogany."
paragraph
5
substring
"leaning on his mahogany cane"
location
extractedText
"leaning on his mahogany cane"
inBounds
true
expectedText
"leaning on his mahogany cane"
locationAccurate
true
14
expected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
status
"matched"
matchedBy
entry
"4415"
detail
"eyeColor"
explanation
"Lucien's eyes are heterochromatic (one amber, one black), not both amber."
paragraph
5
substring
"Both his amber eyes"
location
extractedText
"Both his amber eyes"
inBounds
true
expectedText
"Both his amber eyes were fixed"
locationAccurate
true
15
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
status
"missed"
16
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
status
"matched"
matchedBy
entry
"4423"
detail
"tattoo"
explanation
"Wei-Lin's dragon tattoo covers his entire back, not his chest."
paragraph
7
substring
"The edge of the dragon tattoo on his chest"
location
extractedText
"The edge of the dragon tattoo on his chest"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
true
17
expected
entry
"2006"
detail
"material"
summary
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
status
"missed"
18
expected
entry
"856"
detail
"currentLocation"
summary
"The Veil Market is under Camden per codex, but Wei-Lin says Shoreditch."
status
"matched"
matchedBy
entry
"856"
detail
"currentLocation"
explanation
"The Veil Market is currently beneath Camden, not Shoreditch."
paragraph
7
substring
"abandoned station under Shoreditch"
location
extractedText
"abandoned station under Shoreditch"
inBounds
true
expectedText
"abandoned station under Shoreditch"
locationAccurate
true
19
expected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
status
"matched"
matchedBy
entry
"4420"
detail
"hair"
explanation
"Kai's hair is dyed electric-blue in an undercut, not natural black."
paragraph
8
substring
"natural black hair"
location
extractedText
"natural black hair"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
true
20
expected
entry
"6006"
detail
"logo"
summary
"Blackthorn Security's logo is a thorned branch inside a shield per codex, but Kai says a sword inside a circle."
status
"matched"
matchedBy
entry
"6006"
detail
"logo"
explanation
"Blackthorn Security's logo is a thorned branch inside a shield, not a sword inside a circle."
paragraph
8
substring
"sword inside a circle"
location
extractedText
"sword inside a circle"
inBounds
true
expectedText
"the sword inside a circle"
locationAccurate
true
21
expected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn stick per codex, but the passage says oak."
status
"matched"
matchedBy
entry
"4419"
detail
"walkingStick"
explanation
"Maggie's walking stick is made of blackthorn, not oak."
paragraph
9
substring
"oak walking stick"
location
extractedText
"oak walking stick"
inBounds
true
expectedText
"her oak walking stick"
locationAccurate
true
22
expected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green."
status
"matched"
matchedBy
entry
"4419"
detail
"eyeColor"
explanation
"Maggie's eyes are sharp grey, not green."
paragraph
9
substring
"keen green eyes"
location
extractedText
"keen green eyes"
inBounds
true
expectedText
"Her keen green eyes"
locationAccurate
true
23
expected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Maggie says 1066."
status
"missed"
24
expected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
status
"matched"
matchedBy
entry
"4417"
detail
"scar"
explanation
"Tomás's scar is on his left forearm, not the right."
paragraph
11
substring
"scarred right forearm"
location
extractedText
"[not found in paragraph 11]"
inBounds
true
expectedText
"His scarred right forearm"
locationAccurate
false
25
expected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
status
"matched"
matchedBy
entry
"4417"
detail
"birthplace"
explanation
"Tomás was born in Seville, not Madrid (accent should reflect Seville, not Madrid)."
paragraph
11
substring
"Madrid accent"
location
extractedText
"[not found in paragraph 11]"
inBounds
true
expectedText
"His Madrid accent"
locationAccurate
false
26
expected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue."
status
"matched"
matchedBy
entry
"4418"
detail
"eyeColor"
explanation
"Nyx's eyes are faintly glowing violet, not blue."
paragraph
12
substring
"glowing blue eyes"
location
extractedText
"[not found in paragraph 12]"
inBounds
true
expectedText
"their glowing blue eyes"
locationAccurate
false
27
expected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
status
"matched"
matchedBy
entry
"2001"
detail
"chainMaterial"
explanation
"The Heartstone Pendant's chain is silver, not golden."
paragraph
13
substring
"golden chain"
location
extractedText
"[not found in paragraph 13]"
inBounds
true
expectedText
"the golden chain tangled in her fingers"
locationAccurate
false
28
expected
entry
"2001"
detail
"origin"
summary
"The Heartstone Pendant was forged in Dymas per codex, but Isolde says it was forged in Avaros."
status
"missed"
nearMiss
parsed
entry
"2001"
detail
"gemstoneColor"
explanation
"The Heartstone Pendant's gemstone is deep crimson (correct), but the codex does not mention it glowing brighter than she'd ever seen (unstated detail)."
"Detective Harlow Quinn's watch is on her left wrist, but she is not present in this scene (no violation)."
paragraph
10
substring
"left wrist"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
""
locationAccurate
false
5
parsed
entry
"4412"
detail
"eyeColor"
explanation
"Aurora's eyes are bright blue (correct), but the codex does not mention the Heartstone Pendant glowing brighter than she'd ever seen (unstated detail)."
paragraph
13
substring
"bright blue eyes"
status
"false_positive"
location
extractedText
"[not found in paragraph 13]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
reason
"entry mismatch: got \"4412\", expected \"4424\""
6
parsed
entry
"2001"
detail
"gemstoneColor"
explanation
"The Heartstone Pendant's gemstone is deep crimson (correct), but the codex does not mention it glowing brighter than she'd ever seen (unstated detail)."
paragraph
13
substring
"crimson stone"
status
"false_positive"
location
extractedText
"[not found in paragraph 13]"
inBounds
true
expectedText
"the golden chain tangled in her fingers"
locationAccurate
false
closestExpected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
"Rift points tend to form near underground water sources (correct), but the codex does not specify a heartbeat-like pulsing rhythm (unstated detail)."
paragraph
12
substring
"underground water sources"
status
"false_positive"
location
extractedText
"[not found in paragraph 12]"
inBounds
true
expectedText
""
locationAccurate
false
9
parsed
entry
"4413"
detail
"limp"
explanation
"Silas's limp is in his left leg (correct), but the codex does not mention him rubbing it (unstated detail)."
paragraph
15
substring
"rubbed his left knee"
status
"false_positive"
location
extractedText
"rubbed his left knee"
inBounds
true
expectedText
""
locationAccurate
true
10
parsed
entry
"4413"
detail
"backstory"
explanation
"Silas's botched operation in Prague occurred in an unspecified year, not 1987 (no exact date given in codex)."
paragraph
15
substring
"Prague, 1987"1/1/1987, 12:00:00 AM
status
"false_positive"
location
extractedText
"Prague, 1987"1/1/1987, 12:00:00 AM
inBounds
true
expectedText
""
locationAccurate
true
11
parsed
entry
"7001"
detail
"solsticeEffect"
explanation
"The Veil weakens during the winter solstice, not before it (timing is unspecified in the codex, but \"before\" implies a lead-up period not mentioned)."
paragraph
18
substring
"before the solstice"
status
"false_positive"
location
extractedText
"[not found in paragraph 18]"
inBounds
true
expectedText
""
locationAccurate
false
58.6207%
Precision
True positives: 17/29
totalReported
29
truePositives
17
reportedDetails
0
parsed
entry
"861"
detail
"parkLocation"
explanation
"The Fae Grove is located in Richmond Park, not Hyde Park."
paragraph
1
substring
"Hyde Park"
status
"true_positive"
matchedExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
location
extractedText
"Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
parsed
entry
"4424"
detail
"eyeColor"
explanation
"Isolde's eye color is pale lavender, not pale blue."
paragraph
1
substring
"pale blue eyes"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
location
extractedText
"pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
true
2
parsed
entry
"4424"
detail
"footprints"
explanation
"Isolde leaves no footprints when she walks."
paragraph
1
substring
"bare feet leaving clear prints in the soft earth"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
location
extractedText
"[not found in paragraph 1]"
inBounds
true
expectedText
"her bare feet leaving clear prints in the soft earth"
locationAccurate
false
3
parsed
entry
"4412"
detail
"hair"
explanation
"Aurora's hair is straight shoulder-length black, not silver."
paragraph
2
substring
"her waist-length silver hair"
status
"false_positive"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
false
closestExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
reason
"entry mismatch: got \"4412\", expected \"4420\""
4
parsed
entry
"4412"
detail
"hair"
explanation
"The Fae-Forged Blade is a slender, leaf-shaped dagger, not a hip-worn weapon (no mention of warmth in the codex)."
paragraph
2
substring
"the Fae-Forged Blade warm at her hip"
status
"false_positive"
location
extractedText
"the Fae-Forged Blade warm at her hip"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
false
closestExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
reason
"entry mismatch: got \"4412\", expected \"4420\""
5
parsed
entry
"4414"
detail
"glasses"
explanation
"Eva's glasses are round, not square."
paragraph
2
substring
"square glasses"
status
"true_positive"
matchedExpected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
location
extractedText
"square glasses"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
true
6
parsed
entry
"4414"
detail
"hair"
explanation
"Eva's hair is curly red, but the codex does not mention a wool hat (no contradiction, but the hat is an unstated detail)."
paragraph
2
substring
"curly red hair escaping from under a wool hat"
status
"false_positive"
location
extractedText
"curly red hair escaping from under a wool hat"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
false
closestExpected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
"Maggie has grey eyes per codex, but the passage says green."
location
extractedText
"keen green eyes"
inBounds
true
expectedText
"Her keen green eyes"
locationAccurate
true
13
parsed
entry
"4419"
detail
"walkingStick"
explanation
"Maggie's walking stick is made of blackthorn, not oak."
paragraph
9
substring
"oak walking stick"
status
"true_positive"
matchedExpected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn stick per codex, but the passage says oak."
location
extractedText
"oak walking stick"
inBounds
true
expectedText
"her oak walking stick"
locationAccurate
true
14
parsed
entry
"4416"
detail
"watch"
explanation
"Detective Harlow Quinn's watch is on her left wrist, but she is not present in this scene (no violation)."
paragraph
10
substring
"left wrist"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
""
locationAccurate
false
15
parsed
entry
"4417"
detail
"scar"
explanation
"Tomás's scar is on his left forearm, not the right."
paragraph
11
substring
"scarred right forearm"
status
"true_positive"
matchedExpected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
location
extractedText
"[not found in paragraph 11]"
inBounds
true
expectedText
"His scarred right forearm"
locationAccurate
false
16
parsed
entry
"4417"
detail
"birthplace"
explanation
"Tomás was born in Seville, not Madrid (accent should reflect Seville, not Madrid)."
paragraph
11
substring
"Madrid accent"
status
"true_positive"
matchedExpected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
location
extractedText
"[not found in paragraph 11]"
inBounds
true
expectedText
"His Madrid accent"
locationAccurate
false
17
parsed
entry
"4418"
detail
"eyeColor"
explanation
"Nyx's eyes are faintly glowing violet, not blue."
paragraph
12
substring
"glowing blue eyes"
status
"true_positive"
matchedExpected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue."
location
extractedText
"[not found in paragraph 12]"
inBounds
true
expectedText
"their glowing blue eyes"
locationAccurate
false
18
parsed
entry
"4412"
detail
"eyeColor"
explanation
"Aurora's eyes are bright blue (correct), but the codex does not mention the Heartstone Pendant glowing brighter than she'd ever seen (unstated detail)."
paragraph
13
substring
"bright blue eyes"
status
"false_positive"
location
extractedText
"[not found in paragraph 13]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
reason
"entry mismatch: got \"4412\", expected \"4424\""
19
parsed
entry
"2001"
detail
"gemstoneColor"
explanation
"The Heartstone Pendant's gemstone is deep crimson (correct), but the codex does not mention it glowing brighter than she'd ever seen (unstated detail)."
paragraph
13
substring
"crimson stone"
status
"false_positive"
location
extractedText
"[not found in paragraph 13]"
inBounds
true
expectedText
"the golden chain tangled in her fingers"
locationAccurate
false
closestExpected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
"The Covenant of Separation was signed in 1215, not 1066."
paragraph
4
substring
"the old treaty from 1066"
status
"true_positive"
matchedExpected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
location
extractedText
"the old treaty from 1066"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
true
23
parsed
entry
"856"
detail
"currentLocation"
explanation
"The Veil Market is currently beneath Camden, not Shoreditch."
paragraph
7
substring
"abandoned station under Shoreditch"
status
"true_positive"
matchedExpected
entry
"856"
detail
"currentLocation"
summary
"The Veil Market is under Camden per codex, but Wei-Lin says Shoreditch."
location
extractedText
"abandoned station under Shoreditch"
inBounds
true
expectedText
"abandoned station under Shoreditch"
locationAccurate
true
24
parsed
entry
"6006"
detail
"logo"
explanation
"Blackthorn Security's logo is a thorned branch inside a shield, not a sword inside a circle."
paragraph
8
substring
"sword inside a circle"
status
"true_positive"
matchedExpected
entry
"6006"
detail
"logo"
summary
"Blackthorn Security's logo is a thorned branch inside a shield per codex, but Kai says a sword inside a circle."
location
extractedText
"sword inside a circle"
inBounds
true
expectedText
"the sword inside a circle"
locationAccurate
true
25
parsed
entry
"7003"
detail
"formation"
explanation
"Rift points tend to form near underground water sources (correct), but the codex does not specify a heartbeat-like pulsing rhythm (unstated detail)."
paragraph
12
substring
"underground water sources"
status
"false_positive"
location
extractedText
"[not found in paragraph 12]"
inBounds
true
expectedText
""
locationAccurate
false
26
parsed
entry
"4413"
detail
"limp"
explanation
"Silas's limp is in his left leg (correct), but the codex does not mention him rubbing it (unstated detail)."
paragraph
15
substring
"rubbed his left knee"
status
"false_positive"
location
extractedText
"rubbed his left knee"
inBounds
true
expectedText
""
locationAccurate
true
27
parsed
entry
"4413"
detail
"backstory"
explanation
"Silas's botched operation in Prague occurred in an unspecified year, not 1987 (no exact date given in codex)."
paragraph
15
substring
"Prague, 1987"1/1/1987, 12:00:00 AM
status
"false_positive"
location
extractedText
"Prague, 1987"1/1/1987, 12:00:00 AM
inBounds
true
expectedText
""
locationAccurate
true
28
parsed
entry
"7001"
detail
"solsticeEffect"
explanation
"The Veil weakens during the winter solstice, not before it (timing is unspecified in the codex, but \"before\" implies a lead-up period not mentioned)."