"2026-02-16T01:12:09.055472+00:00"2/16/2026, 1:12:09 AM
model
"nvidia/llama-3.1-nemotron-70b-instruct"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
1536
moderation_latency
(null)
generation_time
27116
tokens_prompt
7863
tokens_completion
997
native_tokens_prompt
7503
native_tokens_completion
926
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0101148
router
(null)
provider_responses
0
id
"cmpl-825f19dd713e4c198cbce352ef5b68cb"
status
200
is_byok
false
latency
102
endpoint_id
"d2a33d30-5d41-47d3-a816-1cc067b5a7dd"
provider_name
"DeepInfra"
model_permaslug
"nvidia/llama-3.1-nemotron-70b-instruct"
api_type
"completions"
id
"gen-1771204301-LUk3iltTcYBKQ5znP8OY"
upstream_id
"cmpl-825f19dd713e4c198cbce352ef5b68cb"
total_cost
0.0101148
cache_discount
(null)
upstream_inference_cost
0
provider_name
"DeepInfra"
Evaluation details
Result
Evaluator
Details
Meta Data
5.0000%
Accuracy (recall)
Matched: 5/30, FP: 7
parsedCount
12
matched
5
total
30
falsePositives
7
expectedDetails
0
expected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
status
"matched"
matchedBy
entry
"861"
detail
"location"
explanation
"The text places the Fae Grove in Hyde Park, but the codex (entry 861) specifies its location as Richmond Park."
paragraph
1
substring
"The Fae Grove in Hyde Park"
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
expected
entry
"861"
detail
"standingStones"
summary
"The grove has oak standing stones per codex, but the passage says elm."
status
"missed"
2
expected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
status
"matched"
matchedBy
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eyes as pale blue, but the codex (entry 4424) states her eye color as pale lavender."
paragraph
2
substring
"her pale blue eyes"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
3
expected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
status
"missed"
nearMiss
parsed
entry
"4424"
detail
"hairLength"
explanation
"The text describes Isolde's hair as waist-length, but the codex (entry 4424) states her hair reaches her waist, which might imply a longer length, but more notably, there's no direct contradiction; however, this detail is flag-worthy due to the potential discrepancy in interpretation."
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
status
"missed"
5
expected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
status
"missed"
6
expected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
status
"missed"
7
expected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but Isolde says white wax."
status
"missed"
8
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Isolde says Prince Lilith."
status
"missed"
9
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
status
"matched"
matchedBy
entry
"4422"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4422) states his eye color as pale blue."
paragraph
5
substring
"his dark brown eyes"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
10
expected
entry
"4422"
detail
"rosary"
summary
"Cormac carries a rowan wood rosary per codex, but the passage says cedar."
status
"missed"
11
expected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
status
"matched"
matchedBy
entry
"7002"
detail
"signedYear"
explanation
"The text mentions the Covenant of Separation's signing year as 1066, which matches the codex (entry 7002). **NO VIOLATION** (Initially thought to be an error due to format request but confirmed to be correct upon reevaluation)"
paragraph
6
substring
"the old treaty from 1066"
location
extractedText
"[not found in paragraph 6]"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
false
12
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
status
"missed"
13
expected
entry
"4415"
detail
"cane"
summary
"Lucien carries an ivory-handled cane per codex, but the passage says mahogany."
status
"missed"
14
expected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
status
"missed"
nearMiss
parsed
entry
"4420"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4420 is not the correct entry; the correct one is 4422) states his eye color as pale blue. (Corrected entry number needed)"
paragraph
5
substring
"his dark brown eyes"
reason
"entry mismatch: got \"4420\", expected \"4415\""
15
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
status
"missed"
16
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
status
"missed"
17
expected
entry
"2006"
detail
"material"
summary
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
status
"missed"
18
expected
entry
"856"
detail
"currentLocation"
summary
"The Veil Market is under Camden per codex, but Wei-Lin says Shoreditch."
status
"missed"
19
expected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
status
"missed"
nearMiss
parsed
entry
"4424"
detail
"hairLength"
explanation
"The text describes Isolde's hair as waist-length, but the codex (entry 4424) states her hair reaches her waist, which might imply a longer length, but more notably, there's no direct contradiction; however, this detail is flag-worthy due to the potential discrepancy in interpretation."
paragraph
2
substring
"her waist-length silver hair"
reason
"entry mismatch: got \"4424\", expected \"4420\""
20
expected
entry
"6006"
detail
"logo"
summary
"Blackthorn Security's logo is a thorned branch inside a shield per codex, but Kai says a sword inside a circle."
status
"missed"
21
expected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn stick per codex, but the passage says oak."
status
"missed"
22
expected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green."
status
"missed"
nearMiss
parsed
entry
"4420"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4420 is not the correct entry; the correct one is 4422) states his eye color as pale blue. (Corrected entry number needed)"
paragraph
5
substring
"his dark brown eyes"
reason
"entry mismatch: got \"4420\", expected \"4419\""
23
expected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Maggie says 1066."
status
"missed"
24
expected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
status
"missed"
25
expected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
status
"missed"
26
expected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue."
status
"missed"
nearMiss
parsed
entry
"4420"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4420 is not the correct entry; the correct one is 4422) states his eye color as pale blue. (Corrected entry number needed)"
paragraph
5
substring
"his dark brown eyes"
reason
"entry mismatch: got \"4420\", expected \"4418\""
27
expected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
status
"missed"
28
expected
entry
"2001"
detail
"origin"
summary
"The Heartstone Pendant was forged in Dymas per codex, but Isolde says it was forged in Avaros."
status
"missed"
29
expected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
status
"matched"
matchedBy
entry
"855"
detail
"ruler"
explanation
"The text states Prince Lilith is the master of the Somnian Cult, but the codex (entry 855) indicates Prince Mammon rules over Avaros (context doesn't directly link to Somnia), and more accurately, entry 859 (Somnia) names Prince Asmodeus as the ruler."
paragraph
10
substring
"Prince Lilith"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
falsePositiveDetails
0
parsed
entry
"4424"
detail
"hairLength"
explanation
"The text describes Isolde's hair as waist-length, but the codex (entry 4424) states her hair reaches her waist, which might imply a longer length, but more notably, there's no direct contradiction; however, this detail is flag-worthy due to the potential discrepancy in interpretation."
paragraph
2
substring
"her waist-length silver hair"
status
"false_positive"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4420 is not the correct entry; the correct one is 4422) states his eye color as pale blue. (Corrected entry number needed)"
paragraph
5
substring
"his dark brown eyes"
status
"false_positive"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
false
closestExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
"There is no character entry for Yu-Fei Cheung in the provided codex to verify details about them, so no violations can be identified for this character based on the given text and codex."
paragraph
(null)
substring
"NA"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
3
parsed
entry
"4429"
detail
"NA"
explanation
"Like Yu-Fei Cheung, there's no entry for Helena Voss in the provided codex to verify details, so no violations can be identified for this character based on the given text and codex."
paragraph
(null)
substring
"NA"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
4
parsed
entry
"859"
detail
"ruler"
explanation
"The text incorrectly identifies Prince Lilith with the Somnian Cult, whereas the codex (entry 859) specifies Prince Asmodeus as the master of Somnia."
paragraph
10
substring
"Prince Lilith"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
closestExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
reason
"entry mismatch: got \"859\", expected \"855\""
5
parsed
entry
"7004"
detail
"sealingMethod"
explanation
"The text doesn't directly contradict the sealing method (blood and record in the Prince's domain), but the context (destruction of contract document in Avaros breaking the bond) may imply a misunderstanding of the helbound soul's contract freedom as stated in the codex (entry 7004), which requires the contract's destruction within the realm where it was signed, yet this seems more a narrative detail than a direct factual contradiction."
paragraph
20
substring
"signed in Avaros"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
6
parsed
entry
"7005"
detail
"NA"
explanation
"The text does not provide information that directly contradicts the Shade Transformation lore in the provided context, given the discussion focuses on other topics."
paragraph
(null)
substring
"NA"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
41.6667%
Precision
True positives: 5/12
totalReported
12
truePositives
5
reportedDetails
0
parsed
entry
"861"
detail
"location"
explanation
"The text places the Fae Grove in Hyde Park, but the codex (entry 861) specifies its location as Richmond Park."
paragraph
1
substring
"The Fae Grove in Hyde Park"
status
"true_positive"
matchedExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
parsed
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eyes as pale blue, but the codex (entry 4424) states her eye color as pale lavender."
paragraph
2
substring
"her pale blue eyes"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
2
parsed
entry
"4424"
detail
"hairLength"
explanation
"The text describes Isolde's hair as waist-length, but the codex (entry 4424) states her hair reaches her waist, which might imply a longer length, but more notably, there's no direct contradiction; however, this detail is flag-worthy due to the potential discrepancy in interpretation."
paragraph
2
substring
"her waist-length silver hair"
status
"false_positive"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4420 is not the correct entry; the correct one is 4422) states his eye color as pale blue. (Corrected entry number needed)"
paragraph
5
substring
"his dark brown eyes"
status
"false_positive"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"his natural black hair falling into his face"
locationAccurate
false
closestExpected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
"The text describes Father Cormac's eyes as dark brown, but the codex (entry 4422) states his eye color as pale blue."
paragraph
5
substring
"his dark brown eyes"
status
"true_positive"
matchedExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
5
parsed
entry
"4425"
detail
"NA"
explanation
"There is no character entry for Yu-Fei Cheung in the provided codex to verify details about them, so no violations can be identified for this character based on the given text and codex."
paragraph
(null)
substring
"NA"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
6
parsed
entry
"7002"
detail
"signedYear"
explanation
"The text mentions the Covenant of Separation's signing year as 1066, which matches the codex (entry 7002). **NO VIOLATION** (Initially thought to be an error due to format request but confirmed to be correct upon reevaluation)"
paragraph
6
substring
"the old treaty from 1066"
status
"true_positive"
matchedExpected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
location
extractedText
"[not found in paragraph 6]"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
false
7
parsed
entry
"4429"
detail
"NA"
explanation
"Like Yu-Fei Cheung, there's no entry for Helena Voss in the provided codex to verify details, so no violations can be identified for this character based on the given text and codex."
paragraph
(null)
substring
"NA"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
8
parsed
entry
"855"
detail
"ruler"
explanation
"The text states Prince Lilith is the master of the Somnian Cult, but the codex (entry 855) indicates Prince Mammon rules over Avaros (context doesn't directly link to Somnia), and more accurately, entry 859 (Somnia) names Prince Asmodeus as the ruler."
paragraph
10
substring
"Prince Lilith"
status
"true_positive"
matchedExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
9
parsed
entry
"859"
detail
"ruler"
explanation
"The text incorrectly identifies Prince Lilith with the Somnian Cult, whereas the codex (entry 859) specifies Prince Asmodeus as the master of Somnia."
paragraph
10
substring
"Prince Lilith"
status
"false_positive"
location
extractedText
"[not found in paragraph 10]"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
closestExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
reason
"entry mismatch: got \"859\", expected \"855\""
10
parsed
entry
"7004"
detail
"sealingMethod"
explanation
"The text doesn't directly contradict the sealing method (blood and record in the Prince's domain), but the context (destruction of contract document in Avaros breaking the bond) may imply a misunderstanding of the helbound soul's contract freedom as stated in the codex (entry 7004), which requires the contract's destruction within the realm where it was signed, yet this seems more a narrative detail than a direct factual contradiction."
paragraph
20
substring
"signed in Avaros"
status
"false_positive"
location
extractedText
(null)
inBounds
false
expectedText
""
locationAccurate
(null)
11
parsed
entry
"7005"
detail
"NA"
explanation
"The text does not provide information that directly contradicts the Shade Transformation lore in the provided context, given the discussion focuses on other topics."