"2026-02-16T01:01:45.322522+00:00"2/16/2026, 1:01:45 AM
model
"nvidia/llama-3.1-nemotron-70b-instruct"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
645
moderation_latency
(null)
generation_time
28439
tokens_prompt
7863
tokens_completion
1056
native_tokens_prompt
7503
native_tokens_completion
1022
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.01023
router
(null)
provider_responses
0
id
"cmpl-bc8f27b8457a483fbd04e99a8168a9ab"
status
200
is_byok
false
latency
196
endpoint_id
"d2a33d30-5d41-47d3-a816-1cc067b5a7dd"
provider_name
"DeepInfra"
model_permaslug
"nvidia/llama-3.1-nemotron-70b-instruct"
api_type
"completions"
id
"gen-1771203676-ZLlguQUgLIpYuyKz8Jf1"
upstream_id
"cmpl-bc8f27b8457a483fbd04e99a8168a9ab"
total_cost
0.01023
cache_discount
(null)
upstream_inference_cost
0
provider_name
"DeepInfra"
Evaluation details
Result
Evaluator
Details
Meta Data
6.6667%
Accuracy (recall)
Matched: 6/30, FP: 8
parsedCount
14
matched
6
total
30
falsePositives
8
expectedDetails
0
expected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
status
"matched"
matchedBy
entry
"861"
detail
"parkLocation"
explanation
"The text states the Fae Grove is in Hyde Park, but the codex specifies it is in Richmond Park (ID 861)."
paragraph
1
substring
"The Fae Grove in Hyde Park"
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
expected
entry
"861"
detail
"standingStones"
summary
"The grove has oak standing stones per codex, but the passage says elm."
status
"missed"
2
expected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
status
"missed"
nearMiss
parsed
entry
"4414"
detail
"eyeColor"
explanation
"The text says Isolde's eyes are pale blue, but the codex says pale lavender (ID 4424)."
paragraph
1
substring
"her pale blue eyes, unblinking, settled on the group"
reason
"entry mismatch: got \"4414\", expected \"4424\""
3
expected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
status
"missed"
4
expected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
status
"missed"
nearMiss
parsed
entry
"2006"
detail
"givenTo"
explanation
"The text implies Aurora has possessed the Fae-Forged Blade for some time, but the codex states it was given to her by Isolde as a gift (with no prior possession indicated, ID 2006)."
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
status
"matched"
matchedBy
entry
"2005"
detail
"containerColor"
explanation
"The text describes Dream Dust containers as red glass vials with white wax, but the codex says they are cobalt-blue glass vials sealed with black wax (ID 2005)."
paragraph
9
substring
"stored in little red glass vials, sealed with white wax"
location
extractedText
"[not found in paragraph 9]"
inBounds
true
expectedText
"little red glass vials"
locationAccurate
false
7
expected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but Isolde says white wax."
status
"missed"
8
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Isolde says Prince Lilith."
status
"missed"
9
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
status
"missed"
nearMiss
parsed
entry
"4414"
detail
"eyeColor"
explanation
"The text says Isolde's eyes are pale blue, but the codex says pale lavender (ID 4424)."
paragraph
1
substring
"her pale blue eyes, unblinking, settled on the group"
reason
"entry mismatch: got \"4414\", expected \"4422\""
10
expected
entry
"4422"
detail
"rosary"
summary
"Cormac carries a rowan wood rosary per codex, but the passage says cedar."
status
"missed"
nearMiss
parsed
entry
"4422"
detail
"hairColor"
explanation
"The text correctly states Father Cormac's hair color as ginger; this is confirmed by the codex (ID 4422)."
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
status
"matched"
matchedBy
entry
"7002"
detail
"signedYear"
explanation
"The text correctly states the Covenant's signing year as 1066, but this is confirmed by the codex (ID 7002); however, it's worth noting for context. A more significant violation is the incorrect implication that the Covenant directly prohibits the specific actions mentioned (it generally forbids direct interference, but the text's context implies a more specific prohibition)."
paragraph
5
substring
"the Covenant of Separation — the old treaty from 1066"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
false
12
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
status
"missed"
13
expected
entry
"4415"
detail
"cane"
summary
"Lucien carries an ivory-handled cane per codex, but the passage says mahogany."
status
"missed"
14
expected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
status
"matched"
matchedBy
entry
"4415"
detail
"eyeColor"
explanation
"The text describes Lucien as having two amber eyes, but the codex states his eyes are heterochromatic — one amber, one black (ID 4415)."
paragraph
4
substring
"Both his amber eyes were fixed on Isolde"
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
"Both his amber eyes were fixed"
locationAccurate
false
15
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
status
"missed"
16
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
status
"missed"
nearMiss
parsed
entry
"4423"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eye color as dark brown, but the codex says pale blue (ID 4422)."
paragraph
10
substring
"his dark brown eyes troubled beneath his thinning ginger hair"
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
status
"missed"
nearMiss
parsed
entry
"2006"
detail
"givenTo"
explanation
"The text implies Aurora has possessed the Fae-Forged Blade for some time, but the codex states it was given to her by Isolde as a gift (with no prior possession indicated, ID 2006)."
"The Veil Market is under Camden per codex, but Wei-Lin says Shoreditch."
status
"missed"
nearMiss
parsed
entry
"854"
detail
"location"
explanation
"The text contradicts itself on the Veil Market's location (Camden vs. Shoreditch), but more critically, the codex does not mention either; it states the market is in an abandoned Tube station beneath Camden (ID 856)."
paragraph
14
substring
"The market's under Camden this month / The market's moved to an abandoned station under Shoreditch for this lunar cycle"
reason
"entry mismatch: got \"854\", expected \"856\""
19
expected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
status
"matched"
matchedBy
entry
"4420"
detail
"hairColor"
explanation
"The text describes Kai's hair as natural black, but the codex says it is dyed electric-blue in an undercut (ID 4420)."
paragraph
8
substring
"his natural black hair falling into his face"
location
extractedText
"his natural black hair falling into his face"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
true
20
expected
entry
"6006"
detail
"logo"
summary
"Blackthorn Security's logo is a thorned branch inside a shield per codex, but Kai says a sword inside a circle."
status
"missed"
21
expected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn stick per codex, but the passage says oak."
status
"missed"
22
expected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green."
status
"missed"
nearMiss
parsed
entry
"4414"
detail
"eyeColor"
explanation
"The text says Isolde's eyes are pale blue, but the codex says pale lavender (ID 4424)."
paragraph
1
substring
"her pale blue eyes, unblinking, settled on the group"
reason
"entry mismatch: got \"4414\", expected \"4419\""
23
expected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Maggie says 1066."
status
"missed"
24
expected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
status
"missed"
25
expected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
status
"missed"
26
expected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue."
status
"missed"
nearMiss
parsed
entry
"4414"
detail
"eyeColor"
explanation
"The text says Isolde's eyes are pale blue, but the codex says pale lavender (ID 4424)."
paragraph
1
substring
"her pale blue eyes, unblinking, settled on the group"
reason
"entry mismatch: got \"4414\", expected \"4418\""
27
expected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
status
"matched"
matchedBy
entry
"2001"
detail
"chainMaterial"
explanation
"The text describes the Heartstone Pendant's chain as golden, but the codex specifies it is a silver chain (ID 2001)."
paragraph
2
substring
"the golden chain tangled in her fingers"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"the golden chain tangled in her fingers"
locationAccurate
false
28
expected
entry
"2001"
detail
"origin"
summary
"The Heartstone Pendant was forged in Dymas per codex, but Isolde says it was forged in Avaros."
status
"missed"
29
expected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
status
"missed"
falsePositiveDetails
0
parsed
entry
"4414"
detail
"eyeColor"
explanation
"The text says Isolde's eyes are pale blue, but the codex says pale lavender (ID 4424)."
paragraph
1
substring
"her pale blue eyes, unblinking, settled on the group"
status
"false_positive"
location
extractedText
"her pale blue eyes, unblinking, settled on the group"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
false
closestExpected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
"The text does not mention a password for Yasmin's Apothecary's back room, but the codex specifies \"moonrise\" as the required password (ID 860)."
paragraph
8
substring
"not mentioned in the relevant dialogue"
status
"false_positive"
location
extractedText
"[not found in paragraph 8]"
inBounds
true
expectedText
""
locationAccurate
false
3
parsed
entry
"2006"
detail
"givenTo"
explanation
"The text implies Aurora has possessed the Fae-Forged Blade for some time, but the codex states it was given to her by Isolde as a gift (with no prior possession indicated, ID 2006)."
paragraph
2
substring
"The Fae-Forged Blade warm at her hip"
status
"false_positive"
location
extractedText
"The Fae-Forged Blade warm at her hip"
inBounds
true
expectedText
"the Fae-Forged Blade warm at her hip"
locationAccurate
true
closestExpected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
"The text contradicts itself on the Veil Market's location (Camden vs. Shoreditch), but more critically, the codex does not mention either; it states the market is in an abandoned Tube station beneath Camden (ID 856)."
paragraph
14
substring
"The market's under Camden this month / The market's moved to an abandoned station under Shoreditch for this lunar cycle"
status
"false_positive"
location
extractedText
"[not found in paragraph 14]"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
false
closestExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
reason
"entry mismatch: got \"854\", expected \"861\""
5
parsed
entry
"4423"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eye color as dark brown, but the codex says pale blue (ID 4422)."
paragraph
10
substring
"his dark brown eyes troubled beneath his thinning ginger hair"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
false
closestExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
"The text does not directly imply a violation but mentions Wei-Lin in the context of Chinatown, while the codex confirms Zhou Wei-Lin owns a noodle shop in Chinatown (ID 4423), this is more of a contextual confirmation than a violation."
paragraph
10
substring
"whispers in Chinatown"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
false
closestExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
"The text does not mention a password for Yasmin's Apothecary's back room, but the codex specifies \"moonrise\" as the required password (ID 860)."
paragraph
8
substring
"not mentioned in the relevant dialogue"
status
"false_positive"
location
extractedText
"[not found in paragraph 8]"
inBounds
true
expectedText
""
locationAccurate
false
3
parsed
entry
"2005"
detail
"containerColor"
explanation
"The text describes Dream Dust containers as red glass vials with white wax, but the codex says they are cobalt-blue glass vials sealed with black wax (ID 2005)."
paragraph
9
substring
"stored in little red glass vials, sealed with white wax"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
location
extractedText
"[not found in paragraph 9]"
inBounds
true
expectedText
"little red glass vials"
locationAccurate
false
4
parsed
entry
"2006"
detail
"givenTo"
explanation
"The text implies Aurora has possessed the Fae-Forged Blade for some time, but the codex states it was given to her by Isolde as a gift (with no prior possession indicated, ID 2006)."
paragraph
2
substring
"The Fae-Forged Blade warm at her hip"
status
"false_positive"
location
extractedText
"The Fae-Forged Blade warm at her hip"
inBounds
true
expectedText
"the Fae-Forged Blade warm at her hip"
locationAccurate
true
closestExpected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
"The text contradicts itself on the Veil Market's location (Camden vs. Shoreditch), but more critically, the codex does not mention either; it states the market is in an abandoned Tube station beneath Camden (ID 856)."
paragraph
14
substring
"The market's under Camden this month / The market's moved to an abandoned station under Shoreditch for this lunar cycle"
status
"false_positive"
location
extractedText
"[not found in paragraph 14]"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
false
closestExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
reason
"entry mismatch: got \"854\", expected \"861\""
6
parsed
entry
"7002"
detail
"signedYear"
explanation
"The text correctly states the Covenant's signing year as 1066, but this is confirmed by the codex (ID 7002); however, it's worth noting for context. A more significant violation is the incorrect implication that the Covenant directly prohibits the specific actions mentioned (it generally forbids direct interference, but the text's context implies a more specific prohibition)."
paragraph
5
substring
"the Covenant of Separation — the old treaty from 1066"
status
"true_positive"
matchedExpected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
false
7
parsed
entry
"4423"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eye color as dark brown, but the codex says pale blue (ID 4422)."
paragraph
10
substring
"his dark brown eyes troubled beneath his thinning ginger hair"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
false
closestExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
"The text states the Fae Grove is in Hyde Park, but the codex specifies it is in Richmond Park (ID 861)."
paragraph
1
substring
"The Fae Grove in Hyde Park"
status
"true_positive"
matchedExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
10
parsed
entry
"4415"
detail
"eyeColor"
explanation
"The text describes Lucien as having two amber eyes, but the codex states his eyes are heterochromatic — one amber, one black (ID 4415)."
paragraph
4
substring
"Both his amber eyes were fixed on Isolde"
status
"true_positive"
matchedExpected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
"Both his amber eyes were fixed"
locationAccurate
false
11
parsed
entry
"4420"
detail
"hairColor"
explanation
"The text describes Kai's hair as natural black, but the codex says it is dyed electric-blue in an undercut (ID 4420)."
paragraph
8
substring
"his natural black hair falling into his face"
status
"true_positive"
matchedExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
location
extractedText
"his natural black hair falling into his face"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
true
12
parsed
entry
"2001"
detail
"chainMaterial"
explanation
"The text describes the Heartstone Pendant's chain as golden, but the codex specifies it is a silver chain (ID 2001)."
paragraph
2
substring
"the golden chain tangled in her fingers"
status
"true_positive"
matchedExpected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"the golden chain tangled in her fingers"
locationAccurate
false
13
parsed
entry
"4423"
detail
"locationOfBusiness"
explanation
"The text does not directly imply a violation but mentions Wei-Lin in the context of Chinatown, while the codex confirms Zhou Wei-Lin owns a noodle shop in Chinatown (ID 4423), this is more of a contextual confirmation than a violation."
paragraph
10
substring
"whispers in Chinatown"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"the dragon tattoo on his chest"
locationAccurate
false
closestExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."