"2026-02-16T00:38:42.66223+00:00"2/16/2026, 12:38:42 AM
model
"nvidia/llama-3.1-nemotron-70b-instruct"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
956
moderation_latency
(null)
generation_time
18180
tokens_prompt
6618
tokens_completion
654
native_tokens_prompt
6419
native_tokens_completion
633
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0084624
router
(null)
provider_responses
0
id
"cmpl-aa95b602209c46e8ad6133be8975e98b"
status
200
is_byok
false
latency
118
endpoint_id
"d2a33d30-5d41-47d3-a816-1cc067b5a7dd"
provider_name
"DeepInfra"
model_permaslug
"nvidia/llama-3.1-nemotron-70b-instruct"
api_type
"completions"
id
"gen-1771202303-YtZPIYcHcixTyMylbjfe"
upstream_id
"cmpl-aa95b602209c46e8ad6133be8975e98b"
total_cost
0.0084624
cache_discount
(null)
upstream_inference_cost
0
provider_name
"DeepInfra"
Evaluation details
Result
Evaluator
Details
Meta Data
50.0000%
Accuracy (recall)
Matched: 6/10, FP: 2
parsedCount
8
matched
6
total
10
falsePositives
2
expectedDetails
0
expected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per the codex, but the passage places it in Hyde Park."
status
"missed"
1
expected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per the codex, but the passage says pale blue eyes."
status
"matched"
matchedBy
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eye color as pale blue, but the codex specifies it as pale lavender (entry 4424, eyeColor)."
paragraph
1
substring
"her pale blue eyes"
location
extractedText
"her pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes settled on Aurora"
locationAccurate
true
2
expected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue glass vials per the codex, but Isolde says red glass vials."
status
"matched"
matchedBy
entry
"2005"
detail
"container color of Dream Dust"
explanation
"The text describes Dream Dust as being stored in little red glass vials, but the codex specifies the containers are cobalt-blue glass vials (entry 2005, containerColor)."
paragraph
2
substring
"stored in little red glass vials, they tell me"
location
extractedText
"stored in little red glass vials, they tell me"
inBounds
true
expectedText
"little red glass vials"
locationAccurate
true
3
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per the codex, but Isolde says their master is Prince Lilith."
status
"missed"
4
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per the codex, but the passage says dark brown eyes."
status
"matched"
matchedBy
entry
"4422"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eye color as dark brown, but the codex specifies it as pale blue (entry 4422, eyeColor)."
paragraph
2
substring
"his dark brown eyes"
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
5
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard is headquartered at St. Brigid's Church, Islington per the codex, but Cormac says their cathedral in Southwark."
status
"missed"
6
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per the codex, but the passage says right arm."
status
"matched"
matchedBy
entry
"4421"
detail
"tattoo"
explanation
"The text places Yasmin's flame serpent tattoo on her right arm, but the codex specifies it as coiling up her left arm (entry 4421, tattoo)."
paragraph
3
substring
"the flame serpent tattoo on her right arm"
location
extractedText
"[not found in paragraph 3]"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
false
7
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his entire back per the codex, but the passage places it on his chest."
status
"matched"
matchedBy
entry
"4423"
detail
"visibility of dragon tattoo"
explanation
"The text implies Wei-Lin's dragon tattoo is partially visible, but the codex describes the tattoo as covering his entire back (entry 4423, tattoo), which wouldn't be visible in this manner."
paragraph
4
substring
"The dragon on his chest was partially visible beneath his open collar"
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
"The dragon on his chest"
locationAccurate
false
8
expected
entry
"2006"
detail
"material"
summary
"The Fae-Forged Blade is made of moonsilver per the codex, but Wei-Lin says bronze daggers."
status
"matched"
matchedBy
entry
"2006"
detail
"material of Fae-Forged Blade"
explanation
"The text implies the Fae-Forged Blade is warm to the touch, but the codex describes it as always cold to the touch (entry 2006, temperature)."
paragraph
5
substring
"the Fae-Forged Blade at her hip. It was warm to the touch, as always"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"bronze daggers that can cut through wards"
locationAccurate
false
9
expected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold to the touch per the codex, but Aurora feels it warm."
status
"missed"
falsePositiveDetails
0
parsed
entry
"4424"
detail
"location of The Fae Grove"
explanation
"The text locates The Fae Grove in Hyde Park, but the codex specifies it's in Richmond Park (entry 861, parkLocation)."
paragraph
1
substring
"The Fae Grove in Hyde Park"
status
"false_positive"
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"Her pale blue eyes settled on Aurora"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per the codex, but the passage says pale blue eyes."
reason
"detail mismatch: got \"location of The Fae Grove\", expected \"eyeColor\""
1
parsed
entry
"4421"
detail
"ruler of Luxuria"
explanation
"The text associates Prince Lilith with Luxuria (implied by context) and the Somnian Cult, but the codex specifies Prince Lilith as the ruler of Luxuria (entry 862, ruler) and Prince Asmodeus as the master of the Somnian Cult (entry 6003, master), with the cult being associated with Somnia (entry 859)."
paragraph
2
substring
"Their master, Prince Lilith, demands more dreamers"
status
"false_positive"
location
extractedText
"Their master, Prince Lilith, demands more dreamers"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
false
closestExpected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per the codex, but the passage says right arm."
reason
"detail mismatch: got \"ruler of Luxuria\", expected \"tattoo\""
75.0000%
Precision
True positives: 6/8
totalReported
8
truePositives
6
reportedDetails
0
parsed
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eye color as pale blue, but the codex specifies it as pale lavender (entry 4424, eyeColor)."
paragraph
1
substring
"her pale blue eyes"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per the codex, but the passage says pale blue eyes."
location
extractedText
"her pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes settled on Aurora"
locationAccurate
true
1
parsed
entry
"4422"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eye color as dark brown, but the codex specifies it as pale blue (entry 4422, eyeColor)."
paragraph
2
substring
"his dark brown eyes"
status
"true_positive"
matchedExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per the codex, but the passage says dark brown eyes."
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
2
parsed
entry
"4421"
detail
"tattoo"
explanation
"The text places Yasmin's flame serpent tattoo on her right arm, but the codex specifies it as coiling up her left arm (entry 4421, tattoo)."
paragraph
3
substring
"the flame serpent tattoo on her right arm"
status
"true_positive"
matchedExpected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per the codex, but the passage says right arm."
location
extractedText
"[not found in paragraph 3]"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
false
3
parsed
entry
"4423"
detail
"visibility of dragon tattoo"
explanation
"The text implies Wei-Lin's dragon tattoo is partially visible, but the codex describes the tattoo as covering his entire back (entry 4423, tattoo), which wouldn't be visible in this manner."
paragraph
4
substring
"The dragon on his chest was partially visible beneath his open collar"
status
"true_positive"
matchedExpected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his entire back per the codex, but the passage places it on his chest."
location
extractedText
"[not found in paragraph 4]"
inBounds
true
expectedText
"The dragon on his chest"
locationAccurate
false
4
parsed
entry
"4424"
detail
"location of The Fae Grove"
explanation
"The text locates The Fae Grove in Hyde Park, but the codex specifies it's in Richmond Park (entry 861, parkLocation)."
paragraph
1
substring
"The Fae Grove in Hyde Park"
status
"false_positive"
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"Her pale blue eyes settled on Aurora"
locationAccurate
false
closestExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per the codex, but the passage says pale blue eyes."
reason
"detail mismatch: got \"location of The Fae Grove\", expected \"eyeColor\""
5
parsed
entry
"2006"
detail
"material of Fae-Forged Blade"
explanation
"The text implies the Fae-Forged Blade is warm to the touch, but the codex describes it as always cold to the touch (entry 2006, temperature)."
paragraph
5
substring
"the Fae-Forged Blade at her hip. It was warm to the touch, as always"
status
"true_positive"
matchedExpected
entry
"2006"
detail
"material"
summary
"The Fae-Forged Blade is made of moonsilver per the codex, but Wei-Lin says bronze daggers."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"bronze daggers that can cut through wards"
locationAccurate
false
6
parsed
entry
"4421"
detail
"ruler of Luxuria"
explanation
"The text associates Prince Lilith with Luxuria (implied by context) and the Somnian Cult, but the codex specifies Prince Lilith as the ruler of Luxuria (entry 862, ruler) and Prince Asmodeus as the master of the Somnian Cult (entry 6003, master), with the cult being associated with Somnia (entry 859)."
paragraph
2
substring
"Their master, Prince Lilith, demands more dreamers"
status
"false_positive"
location
extractedText
"Their master, Prince Lilith, demands more dreamers"
inBounds
true
expectedText
"tattoo on her right arm"
locationAccurate
false
closestExpected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per the codex, but the passage says right arm."
reason
"detail mismatch: got \"ruler of Luxuria\", expected \"tattoo\""
7
parsed
entry
"2005"
detail
"container color of Dream Dust"
explanation
"The text describes Dream Dust as being stored in little red glass vials, but the codex specifies the containers are cobalt-blue glass vials (entry 2005, containerColor)."
paragraph
2
substring
"stored in little red glass vials, they tell me"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue glass vials per the codex, but Isolde says red glass vials."