"2026-02-16T00:53:48.375888+00:00"2/16/2026, 12:53:48 AM
model
"meta-llama/llama-3.1-8b-instruct"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
249
moderation_latency
(null)
generation_time
1376
tokens_prompt
2721
tokens_completion
847
native_tokens_prompt
2467
native_tokens_completion
818
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.00018879
router
(null)
provider_responses
0
id
"chatcmpl-3f2fe2a1-483c-480d-be6c-758e4ac6f6ff"
status
200
is_byok
false
latency
102
endpoint_id
"74af5aaf-6ad5-4389-ae0f-1bd2c7337386"
provider_name
"Groq"
model_permaslug
"meta-llama/llama-3.1-8b-instruct"
api_type
"completions"
id
"gen-1771203226-cuUa0SZHJDGc0xlNMmHj"
upstream_id
"chatcmpl-3f2fe2a1-483c-480d-be6c-758e4ac6f6ff"
total_cost
0.00018879
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Groq"
Evaluation details
Result
Evaluator
Details
Meta Data
38.8889%
Accuracy (recall)
Matched: 5/9, FP: 3
parsedCount
8
matched
5
total
9
falsePositives
3
expectedDetails
0
expected
entry
"854"
detail
"signColor"
summary
"The Raven's Nest has a green neon sign per codex, but the passage says red."
status
"missed"
1
expected
entry
"4413"
detail
"beard"
summary
"Silas's beard matches his grey-streaked auburn hair per codex, but the passage says dark brown beard."
status
"missed"
nearMiss
parsed
entry
"4413"
detail
"height"
explanation
"The text does not reference this however the id \"4413\" belongs to Silas, and codex lists him as 6'1\" but there are no other characters listed with this ID as Silas, this could be Silas."
paragraph
2
substring
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
"Dymas is Prince Belphegor's realm per codex, but Rory refers to it as Prince Mammon's territory."
status
"matched"
matchedBy
entry
"853"
detail
"ruler"
explanation
"The text says the ruler of Dymas is Prince Mammon, but the codex says Prince Belphegor is the correct ruler of Dymas (Gluttony)."
paragraph
5
substring
"Rory crossed over through the rift point under the Thames — the one near Blackfriars. Prince Mammon's territory was... different. The vineyards were withered. The amber sky had turned grey"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"Prince Mammon's territory"
locationAccurate
false
5
expected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Rory thinks 1066."
status
"matched"
matchedBy
entry
"6001"
detail
"foundingYear"
explanation
"The text says the Wardens were founded in 1066, but the codex says they were founded in 1215."
paragraph
5
substring
"The Wardens will lock it in the Westminster Catacombs and pretend it doesn't exist, their founding in 1066 gave them centuries of institutional arrogance"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"Their founding in 1066"
locationAccurate
false
6
expected
entry
"4413"
detail
"limp"
summary
"Silas limps on his left leg per codex, but the passage says right leg."
status
"missed"
nearMiss
parsed
entry
"4413"
detail
"height"
explanation
"The text does not reference this however the id \"4413\" belongs to Silas, and codex lists him as 6'1\" but there are no other characters listed with this ID as Silas, this could be Silas."
paragraph
2
substring
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
"The Veil weakens during the winter solstice per codex, but the passage says summer solstice."
status
"matched"
matchedBy
entry
"7001"
detail
"solsticeEffect"
explanation
"The text says the founding of the Wardens was in 1066, but also says in three days the summer solstice which would be in the year 2026 since 2023 was mentioned, however the text's solstice effect would mean the Veil weakens in winter which is incorrect the codex says it weakens in the winter solstice and strengthens during the summer solstice."
paragraph
5
substring
"the summer solstice is three days away, and the rift activity is spiking"
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"The summer solstice is three days away, and the rift activity is spiking"
locationAccurate
false
8
expected
entry
"4412"
detail
"scar"
summary
"Aurora's crescent scar is on her left wrist per codex, but the passage says right wrist."
status
"matched"
matchedBy
entry
"4412"
detail
"scar"
explanation
"The text says scar is on her left wrist, but the codex says it's on her left hand is wrong it is actually on the left wrist however there isn't information on the location, the codex says left wrist, and also the text says childhood accident while the codex says left wrist from a childhood accident."
paragraph
1
substring
"a detour through Chinatown after a customer on Wardour Street had tipped her off about strange lights near an alley"
location
extractedText
"a detour through Chinatown after a customer on Wardour Street had tipped her off about strange lights near an alley"
inBounds
true
expectedText
"the crescent-shaped scar on her right wrist"
locationAccurate
false
falsePositiveDetails
0
parsed
entry
"4412"
detail
"height"
explanation
"The text says 5'6\", but the codex says 5'6\" is the correct height, but no other character with this ID is referenced. However, the id 4412 does belong to Aurora Carter. The text says \"She\" is 5'6\" but there should be no discrepancy between height and height value, it appears there is a mistake in the text since Aurora is 5'6\"."
paragraph
1
substring
"The Raven's Nest waited. She paused beneath it, catching her breath"
status
"false_positive"
location
extractedText
"[not found in paragraph 1]"
inBounds
true
expectedText
"the crescent-shaped scar on her right wrist"
locationAccurate
false
closestExpected
entry
"4412"
detail
"scar"
summary
"Aurora's crescent scar is on her left wrist per codex, but the passage says right wrist."
"The text does not reference this however the id \"4413\" belongs to Silas, and codex lists him as 6'1\" but there are no other characters listed with this ID as Silas, this could be Silas."
paragraph
2
substring
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
status
"false_positive"
location
extractedText
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
inBounds
true
expectedText
"his dark brown beard had been freshly trimmed"
locationAccurate
false
closestExpected
entry
"4413"
detail
"beard"
summary
"Silas's beard matches his grey-streaked auburn hair per codex, but the passage says dark brown beard."
"The text says 5'6\", but the codex says 5'6\" is the correct height, but no other character with this ID is referenced. However, the id 4412 does belong to Aurora Carter. The text says \"She\" is 5'6\" but there should be no discrepancy between height and height value, it appears there is a mistake in the text since Aurora is 5'6\"."
paragraph
1
substring
"The Raven's Nest waited. She paused beneath it, catching her breath"
status
"false_positive"
location
extractedText
"[not found in paragraph 1]"
inBounds
true
expectedText
"the crescent-shaped scar on her right wrist"
locationAccurate
false
closestExpected
entry
"4412"
detail
"scar"
summary
"Aurora's crescent scar is on her left wrist per codex, but the passage says right wrist."
"The text says scar is on her left wrist, but the codex says it's on her left hand is wrong it is actually on the left wrist however there isn't information on the location, the codex says left wrist, and also the text says childhood accident while the codex says left wrist from a childhood accident."
paragraph
1
substring
"a detour through Chinatown after a customer on Wardour Street had tipped her off about strange lights near an alley"
status
"true_positive"
matchedExpected
entry
"4412"
detail
"scar"
summary
"Aurora's crescent scar is on her left wrist per codex, but the passage says right wrist."
location
extractedText
"a detour through Chinatown after a customer on Wardour Street had tipped her off about strange lights near an alley"
inBounds
true
expectedText
"the crescent-shaped scar on her right wrist"
locationAccurate
false
2
parsed
entry
"4413"
detail
"signetRing"
explanation
"The text says signet ring on the left hand, but the codex says silver signet ring is correct and it should be on the right hand."
paragraph
2
substring
"The silver signet ring on his left hand caught the light as he set down a glass of single malt"
status
"true_positive"
matchedExpected
entry
"4413"
detail
"signetRing"
summary
"Silas wears his signet ring on his right hand per codex, but the passage says left hand."
location
extractedText
"[not found in paragraph 2]"
inBounds
true
expectedText
"The silver signet ring on his left hand"
locationAccurate
false
3
parsed
entry
"853"
detail
"ruler"
explanation
"The text says the ruler of Dymas is Prince Mammon, but the codex says Prince Belphegor is the correct ruler of Dymas (Gluttony)."
paragraph
5
substring
"Rory crossed over through the rift point under the Thames — the one near Blackfriars. Prince Mammon's territory was... different. The vineyards were withered. The amber sky had turned grey"
status
"true_positive"
matchedExpected
entry
"853"
detail
"ruler"
summary
"Dymas is Prince Belphegor's realm per codex, but Rory refers to it as Prince Mammon's territory."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"Prince Mammon's territory"
locationAccurate
false
4
parsed
entry
"7001"
detail
"solsticeEffect"
explanation
"The text says the founding of the Wardens was in 1066, but also says in three days the summer solstice which would be in the year 2026 since 2023 was mentioned, however the text's solstice effect would mean the Veil weakens in winter which is incorrect the codex says it weakens in the winter solstice and strengthens during the summer solstice."
paragraph
5
substring
"the summer solstice is three days away, and the rift activity is spiking"
status
"true_positive"
matchedExpected
entry
"7001"
detail
"solsticeEffect"
summary
"The Veil weakens during the winter solstice per codex, but the passage says summer solstice."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"The summer solstice is three days away, and the rift activity is spiking"
locationAccurate
false
5
parsed
entry
"6001"
detail
"foundingYear"
explanation
"The text says the Wardens were founded in 1066, but the codex says they were founded in 1215."
paragraph
5
substring
"The Wardens will lock it in the Westminster Catacombs and pretend it doesn't exist, their founding in 1066 gave them centuries of institutional arrogance"
status
"true_positive"
matchedExpected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Rory thinks 1066."
location
extractedText
"[not found in paragraph 5]"
inBounds
true
expectedText
"Their founding in 1066"
locationAccurate
false
6
parsed
entry
"2001"
detail
"glow"
explanation
"The text says the glow is an inner glow, but the codex says it has a faint inner glow and it glows brightly when near a Hel portal."
paragraph
6
substring
"The pendant warm against her hip, and headed east toward Brick Lane — toward answers she wasn't sure she wanted to find"
status
"false_positive"
location
extractedText
"[not found in paragraph 6]"
inBounds
true
expectedText
"The golden chain pooled around the crimson stone"
locationAccurate
false
closestExpected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
"The text does not reference this however the id \"4413\" belongs to Silas, and codex lists him as 6'1\" but there are no other characters listed with this ID as Silas, this could be Silas."
paragraph
2
substring
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
status
"false_positive"
location
extractedText
"Inside, the familiar smell of aged wood and whiskey enveloped her. The walls were lined with their usual gallery of old maps and black-and-white photographs — Silas had once told her each one marked a place where the Veil had torn."
inBounds
true
expectedText
"his dark brown beard had been freshly trimmed"
locationAccurate
false
closestExpected
entry
"4413"
detail
"beard"
summary
"Silas's beard matches his grey-streaked auburn hair per codex, but the passage says dark brown beard."