"2026-02-16T10:56:04.628073+00:00"2/16/2026, 10:56:04 AM
model
"qwen/qwen3.5-plus-20260216"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
993
moderation_latency
(null)
generation_time
182096
tokens_prompt
7819
tokens_completion
5259
native_tokens_prompt
7919
native_tokens_completion
5723
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0169028
router
(null)
provider_responses
0
id
"chatcmpl-26cd6e57-85c6-99e4-b02a-8ece0147ca72"
status
200
is_byok
false
latency
993
endpoint_id
"71e0f6c3-7815-4734-92d3-8a04f3b0ca91"
provider_name
"Alibaba"
model_permaslug
"qwen/qwen3.5-plus-20260216"
api_type
"completions"
id
"gen-1771239182-liVp2Jpbe8VWckFjGjxe"
upstream_id
"chatcmpl-26cd6e57-85c6-99e4-b02a-8ece0147ca72"
total_cost
0.0169028
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Alibaba"
Evaluation details
Result
Evaluator
Details
Meta Data
43.3333%
Accuracy (recall)
Matched: 14/30, FP: 2
parsedCount
16
matched
14
total
30
falsePositives
2
expectedDetails
0
expected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
status
"matched"
matchedBy
entry
"861"
detail
"parkLocation"
explanation
"The text places the grove in Hyde Park, but the codex states it is in Richmond Park."
paragraph
1
substring
"The Fae Grove in Hyde Park"
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
expected
entry
"861"
detail
"standingStones"
summary
"The grove has oak standing stones per codex, but the passage says elm."
status
"matched"
matchedBy
entry
"861"
detail
"standingStones"
explanation
"The text describes the stones as elm, but the codex specifies they are ancient oak standing stones."
paragraph
1
substring
"ancient elm standing stones"
location
extractedText
"ancient elm standing stones"
inBounds
true
expectedText
"The ancient elm standing stones"
locationAccurate
true
2
expected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
status
"matched"
matchedBy
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eyes as pale blue, but the codex says they are pale lavender."
paragraph
1
substring
"Her pale blue eyes"
location
extractedText
"Her pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
true
3
expected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
status
"matched"
matchedBy
entry
"4424"
detail
"footprints"
explanation
"The text says Isolde leaves prints, but the codex states she leaves no footprints when she walks."
paragraph
3
substring
"leaving clear prints in the soft earth"
location
extractedText
"leaving clear prints in the soft earth"
inBounds
true
expectedText
"her bare feet leaving clear prints in the soft earth"
locationAccurate
true
4
expected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
status
"matched"
matchedBy
entry
"2006"
detail
"temperature"
explanation
"The text says the blade is warm, but the codex states it is always cold to the touch."
paragraph
2
substring
"Fae-Forged Blade warm at her hip"
location
extractedText
"Fae-Forged Blade warm at her hip"
inBounds
true
expectedText
"the Fae-Forged Blade warm at her hip"
locationAccurate
true
5
expected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
status
"matched"
matchedBy
entry
"4414"
detail
"glasses"
explanation
"The text describes Eva's glasses as square, but the codex says they are round."
paragraph
2
substring
"Her square glasses"
location
extractedText
"Her square glasses"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
true
6
expected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
status
"matched"
matchedBy
entry
"2005"
detail
"containerColor"
explanation
"The text says the vials are red, but the codex states Dream Dust is stored in cobalt-blue glass vials."
paragraph
3
substring
"little red glass vials"
location
extractedText
"little red glass vials"
inBounds
true
expectedText
"little red glass vials"
locationAccurate
true
7
expected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but Isolde says white wax."
status
"matched"
matchedBy
entry
"2005"
detail
"sealMaterial"
explanation
"The text says the seal is white wax, but the codex states it is sealed with black wax."
paragraph
3
substring
"sealed with white wax"
location
extractedText
"sealed with white wax"
inBounds
true
expectedText
"sealed with white wax"
locationAccurate
true
8
expected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Isolde says Prince Lilith."
status
"matched"
matchedBy
entry
"6003"
detail
"master"
explanation
"The text identifies Lilith as the master of the Somnian Cult, but the codex says the master is Prince Asmodeus."
paragraph
3
substring
"Prince Lilith"
location
extractedText
"Prince Lilith"
inBounds
true
expectedText
"Prince Lilith, demands more dreamers"
locationAccurate
true
9
expected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
status
"matched"
matchedBy
entry
"4422"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex says they are pale blue."
paragraph
4
substring
"dark brown eyes"
location
extractedText
"dark brown eyes"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
true
10
expected
entry
"4422"
detail
"rosary"
summary
"Cormac carries a rowan wood rosary per codex, but the passage says cedar."
status
"matched"
matchedBy
entry
"4422"
detail
"rosary"
explanation
"The text says the rosary is cedar, but the codex specifies it is made of rowan wood beads."
paragraph
4
substring
"cedar rosary beads"
location
extractedText
"cedar rosary beads"
inBounds
true
expectedText
"his cedar rosary beads"
locationAccurate
true
11
expected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
status
"matched"
matchedBy
entry
"7002"
detail
"signedYear"
explanation
"The text dates the Covenant of Separation to 1066, but the codex says it was signed in 1215."
paragraph
4
substring
"treaty from 1066"
location
extractedText
"treaty from 1066"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
true
12
expected
entry
"6004"
detail
"headquarters"
summary
"The Parish Guard HQ is St. Brigid's in Islington per codex, but Cormac says Southwark."
status
"missed"
13
expected
entry
"4415"
detail
"cane"
summary
"Lucien carries an ivory-handled cane per codex, but the passage says mahogany."
status
"matched"
matchedBy
entry
"4415"
detail
"cane"
explanation
"The text describes the cane as mahogany, but the codex says it is an ivory-handled cane."
paragraph
5
substring
"mahogany cane"
location
extractedText
"mahogany cane"
inBounds
true
expectedText
"leaning on his mahogany cane"
locationAccurate
true
14
expected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
status
"matched"
matchedBy
entry
"4415"
detail
"eyeColor"
explanation
"The text says both of Lucien's eyes are amber, but the codex states he has heterochromatic eyes (one amber, one black)."
paragraph
5
substring
"Both his amber eyes"
location
extractedText
"Both his amber eyes"
inBounds
true
expectedText
"Both his amber eyes were fixed"
locationAccurate
true
15
expected
entry
"4421"
detail
"tattoo"
summary
"Yasmin's flame serpent tattoo is on her left arm per codex, but the passage says right arm."
status
"missed"
16
expected
entry
"4423"
detail
"tattoo"
summary
"Wei-Lin's dragon tattoo covers his back per codex, but the passage says chest."
status
"missed"
17
expected
entry
"2006"
detail
"material"
summary
"Fae-Forged Blades are moonsilver per codex, but Wei-Lin says bronze."
status
"missed"
18
expected
entry
"856"
detail
"currentLocation"
summary
"The Veil Market is under Camden per codex, but Wei-Lin says Shoreditch."
status
"missed"
19
expected
entry
"4420"
detail
"hair"
summary
"Kai has dyed electric-blue hair per codex, but the passage says natural black hair."
status
"missed"
20
expected
entry
"6006"
detail
"logo"
summary
"Blackthorn Security's logo is a thorned branch inside a shield per codex, but Kai says a sword inside a circle."
status
"missed"
21
expected
entry
"4419"
detail
"walkingStick"
summary
"Maggie carries a blackthorn stick per codex, but the passage says oak."
status
"missed"
22
expected
entry
"4419"
detail
"eyeColor"
summary
"Maggie has grey eyes per codex, but the passage says green."
status
"missed"
23
expected
entry
"6001"
detail
"foundingYear"
summary
"The Wardens were founded in 1215 per codex, but Maggie says 1066."
status
"missed"
24
expected
entry
"4417"
detail
"scar"
summary
"Tomás has a scar on his left forearm per codex, but the passage says right forearm."
status
"missed"
25
expected
entry
"4417"
detail
"birthplace"
summary
"Tomás was born in Seville per codex, but the passage implies Madrid accent."
status
"missed"
26
expected
entry
"4418"
detail
"eyeColor"
summary
"Nyx has violet eyes per codex, but the passage says blue."
status
"missed"
27
expected
entry
"2001"
detail
"chainMaterial"
summary
"The Heartstone Pendant has a silver chain per codex, but the passage says golden chain."
status
"missed"
28
expected
entry
"2001"
detail
"origin"
summary
"The Heartstone Pendant was forged in Dymas per codex, but Isolde says it was forged in Avaros."
status
"missed"
29
expected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
status
"missed"
nearMiss
parsed
entry
"857"
detail
"ruler"
explanation
"The text implies Irath is the new base, but the context of the Crimson Court (demon aristocracy) and the shift mentioned later regarding Belphegor suggests confusion; however, the specific violation here is Lucien's father. Lucien's father is a demon from Avaros (Greed/Mammon), not associated with Irath (Wrath/Abaddon) in a way that fits the 'Crimson Court base' narrative without contradicting the realm rulers. More directly, the text says the base shifted *from* Dymas *to* Irath. While possible, the next violation is clearer. Let's stick to the clear ones. Wait, the text says \"Mammon breaks it daily in Avaros\" which is correct. But then Lucien says his father says the base shifted. Lucien's father is from Avaros. This isn't a direct contradiction yet. Let's look closer. Ah, the Crimson Court headquarters is Dymas. The text says it shifted to Irath. This is a plot point, not necessarily a contradiction unless the codex says it *never* moves. But the codex says HQ is Dymas. Let's look for harder contradictions.\nActually, the text says \"Prince Lilith\" rules the Somnian Cult. That is a hard contradiction (Asmodeus).\nLet's re-evaluate the \"base of operations\" line. The codex says Crimson Court HQ is Dymas. The text says it shifted. This might be a story development, but if the codex defines the HQ as Dymas, stating it has moved to Irath without codex support for the move might be a violation of the static \"headquarters\" field. However, usually, story movement overrides static bibles. Let's skip this one as ambiguous unless there's a \"cannot move\" rule.\nLet's look at Yasmin."
paragraph
5
substring
"shifted from Dymas to Irath"
reason
"entry mismatch: got \"857\", expected \"855\""
falsePositiveDetails
0
parsed
entry
"4422"
detail
"description"
explanation
"The text places Father Cormac's cathedral in Southwark, but the codex assigns him to St. Brigid's parish in Islington."
paragraph
4
substring
"cathedral in Southwark"
status
"false_positive"
location
extractedText
"cathedral in Southwark"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
closestExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
"The text implies Irath is the new base, but the context of the Crimson Court (demon aristocracy) and the shift mentioned later regarding Belphegor suggests confusion; however, the specific violation here is Lucien's father. Lucien's father is a demon from Avaros (Greed/Mammon), not associated with Irath (Wrath/Abaddon) in a way that fits the 'Crimson Court base' narrative without contradicting the realm rulers. More directly, the text says the base shifted *from* Dymas *to* Irath. While possible, the next violation is clearer. Let's stick to the clear ones. Wait, the text says \"Mammon breaks it daily in Avaros\" which is correct. But then Lucien says his father says the base shifted. Lucien's father is from Avaros. This isn't a direct contradiction yet. Let's look closer. Ah, the Crimson Court headquarters is Dymas. The text says it shifted to Irath. This is a plot point, not necessarily a contradiction unless the codex says it *never* moves. But the codex says HQ is Dymas. Let's look for harder contradictions.\nActually, the text says \"Prince Lilith\" rules the Somnian Cult. That is a hard contradiction (Asmodeus).\nLet's re-evaluate the \"base of operations\" line. The codex says Crimson Court HQ is Dymas. The text says it shifted. This might be a story development, but if the codex defines the HQ as Dymas, stating it has moved to Irath without codex support for the move might be a violation of the static \"headquarters\" field. However, usually, story movement overrides static bibles. Let's skip this one as ambiguous unless there's a \"cannot move\" rule.\nLet's look at Yasmin."
paragraph
5
substring
"shifted from Dymas to Irath"
status
"false_positive"
location
extractedText
"shifted from Dymas to Irath"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
closestExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."
reason
"entry mismatch: got \"857\", expected \"855\""
87.5000%
Precision
True positives: 14/16
totalReported
16
truePositives
14
reportedDetails
0
parsed
entry
"861"
detail
"parkLocation"
explanation
"The text places the grove in Hyde Park, but the codex states it is in Richmond Park."
paragraph
1
substring
"The Fae Grove in Hyde Park"
status
"true_positive"
matchedExpected
entry
"861"
detail
"parkLocation"
summary
"The Fae Grove is in Richmond Park per codex, but the passage says Hyde Park."
location
extractedText
"The Fae Grove in Hyde Park"
inBounds
true
expectedText
"The Fae Grove in Hyde Park"
locationAccurate
true
1
parsed
entry
"861"
detail
"standingStones"
explanation
"The text describes the stones as elm, but the codex specifies they are ancient oak standing stones."
paragraph
1
substring
"ancient elm standing stones"
status
"true_positive"
matchedExpected
entry
"861"
detail
"standingStones"
summary
"The grove has oak standing stones per codex, but the passage says elm."
location
extractedText
"ancient elm standing stones"
inBounds
true
expectedText
"The ancient elm standing stones"
locationAccurate
true
2
parsed
entry
"4424"
detail
"eyeColor"
explanation
"The text describes Isolde's eyes as pale blue, but the codex says they are pale lavender."
paragraph
1
substring
"Her pale blue eyes"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"eyeColor"
summary
"Isolde has pale lavender eyes per codex, but the passage says pale blue eyes."
location
extractedText
"Her pale blue eyes"
inBounds
true
expectedText
"Her pale blue eyes, unblinking"
locationAccurate
true
3
parsed
entry
"2006"
detail
"temperature"
explanation
"The text says the blade is warm, but the codex states it is always cold to the touch."
paragraph
2
substring
"Fae-Forged Blade warm at her hip"
status
"true_positive"
matchedExpected
entry
"2006"
detail
"temperature"
summary
"The Fae-Forged Blade is always cold per codex, but the passage says warm."
location
extractedText
"Fae-Forged Blade warm at her hip"
inBounds
true
expectedText
"the Fae-Forged Blade warm at her hip"
locationAccurate
true
4
parsed
entry
"4414"
detail
"glasses"
explanation
"The text describes Eva's glasses as square, but the codex says they are round."
paragraph
2
substring
"Her square glasses"
status
"true_positive"
matchedExpected
entry
"4414"
detail
"glasses"
summary
"Eva wears round glasses per codex, but the passage says square glasses."
location
extractedText
"Her square glasses"
inBounds
true
expectedText
"Her square glasses kept fogging"
locationAccurate
true
5
parsed
entry
"4424"
detail
"footprints"
explanation
"The text says Isolde leaves prints, but the codex states she leaves no footprints when she walks."
paragraph
3
substring
"leaving clear prints in the soft earth"
status
"true_positive"
matchedExpected
entry
"4424"
detail
"footprints"
summary
"Isolde leaves no footprints per codex, but the passage says she leaves clear prints."
location
extractedText
"leaving clear prints in the soft earth"
inBounds
true
expectedText
"her bare feet leaving clear prints in the soft earth"
locationAccurate
true
6
parsed
entry
"2005"
detail
"containerColor"
explanation
"The text says the vials are red, but the codex states Dream Dust is stored in cobalt-blue glass vials."
paragraph
3
substring
"little red glass vials"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"containerColor"
summary
"Dream Dust is stored in cobalt-blue vials per codex, but Isolde says red glass vials."
location
extractedText
"little red glass vials"
inBounds
true
expectedText
"little red glass vials"
locationAccurate
true
7
parsed
entry
"2005"
detail
"sealMaterial"
explanation
"The text says the seal is white wax, but the codex states it is sealed with black wax."
paragraph
3
substring
"sealed with white wax"
status
"true_positive"
matchedExpected
entry
"2005"
detail
"sealMaterial"
summary
"Dream Dust vials are sealed with black wax per codex, but Isolde says white wax."
location
extractedText
"sealed with white wax"
inBounds
true
expectedText
"sealed with white wax"
locationAccurate
true
8
parsed
entry
"6003"
detail
"master"
explanation
"The text identifies Lilith as the master of the Somnian Cult, but the codex says the master is Prince Asmodeus."
paragraph
3
substring
"Prince Lilith"
status
"true_positive"
matchedExpected
entry
"6003"
detail
"master"
summary
"The Somnian Cult serves Prince Asmodeus per codex, but Isolde says Prince Lilith."
location
extractedText
"Prince Lilith"
inBounds
true
expectedText
"Prince Lilith, demands more dreamers"
locationAccurate
true
9
parsed
entry
"4422"
detail
"eyeColor"
explanation
"The text describes Father Cormac's eyes as dark brown, but the codex says they are pale blue."
paragraph
4
substring
"dark brown eyes"
status
"true_positive"
matchedExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
location
extractedText
"dark brown eyes"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
true
10
parsed
entry
"4422"
detail
"rosary"
explanation
"The text says the rosary is cedar, but the codex specifies it is made of rowan wood beads."
paragraph
4
substring
"cedar rosary beads"
status
"true_positive"
matchedExpected
entry
"4422"
detail
"rosary"
summary
"Cormac carries a rowan wood rosary per codex, but the passage says cedar."
location
extractedText
"cedar rosary beads"
inBounds
true
expectedText
"his cedar rosary beads"
locationAccurate
true
11
parsed
entry
"4422"
detail
"description"
explanation
"The text places Father Cormac's cathedral in Southwark, but the codex assigns him to St. Brigid's parish in Islington."
paragraph
4
substring
"cathedral in Southwark"
status
"false_positive"
location
extractedText
"cathedral in Southwark"
inBounds
true
expectedText
"his dark brown eyes troubled"
locationAccurate
false
closestExpected
entry
"4422"
detail
"eyeColor"
summary
"Father Cormac has pale blue eyes per codex, but the passage says dark brown eyes."
"The text dates the Covenant of Separation to 1066, but the codex says it was signed in 1215."
paragraph
4
substring
"treaty from 1066"
status
"true_positive"
matchedExpected
entry
"7002"
detail
"signedYear"
summary
"The Covenant of Separation was signed in 1215 per codex, but Cormac says 1066."
location
extractedText
"treaty from 1066"
inBounds
true
expectedText
"the old treaty from 1066"
locationAccurate
true
13
parsed
entry
"4415"
detail
"eyeColor"
explanation
"The text says both of Lucien's eyes are amber, but the codex states he has heterochromatic eyes (one amber, one black)."
paragraph
5
substring
"Both his amber eyes"
status
"true_positive"
matchedExpected
entry
"4415"
detail
"eyeColor"
summary
"Lucien has heterochromatic eyes (one amber, one black), but the passage says both amber."
location
extractedText
"Both his amber eyes"
inBounds
true
expectedText
"Both his amber eyes were fixed"
locationAccurate
true
14
parsed
entry
"4415"
detail
"cane"
explanation
"The text describes the cane as mahogany, but the codex says it is an ivory-handled cane."
paragraph
5
substring
"mahogany cane"
status
"true_positive"
matchedExpected
entry
"4415"
detail
"cane"
summary
"Lucien carries an ivory-handled cane per codex, but the passage says mahogany."
location
extractedText
"mahogany cane"
inBounds
true
expectedText
"leaning on his mahogany cane"
locationAccurate
true
15
parsed
entry
"857"
detail
"ruler"
explanation
"The text implies Irath is the new base, but the context of the Crimson Court (demon aristocracy) and the shift mentioned later regarding Belphegor suggests confusion; however, the specific violation here is Lucien's father. Lucien's father is a demon from Avaros (Greed/Mammon), not associated with Irath (Wrath/Abaddon) in a way that fits the 'Crimson Court base' narrative without contradicting the realm rulers. More directly, the text says the base shifted *from* Dymas *to* Irath. While possible, the next violation is clearer. Let's stick to the clear ones. Wait, the text says \"Mammon breaks it daily in Avaros\" which is correct. But then Lucien says his father says the base shifted. Lucien's father is from Avaros. This isn't a direct contradiction yet. Let's look closer. Ah, the Crimson Court headquarters is Dymas. The text says it shifted to Irath. This is a plot point, not necessarily a contradiction unless the codex says it *never* moves. But the codex says HQ is Dymas. Let's look for harder contradictions.\nActually, the text says \"Prince Lilith\" rules the Somnian Cult. That is a hard contradiction (Asmodeus).\nLet's re-evaluate the \"base of operations\" line. The codex says Crimson Court HQ is Dymas. The text says it shifted. This might be a story development, but if the codex defines the HQ as Dymas, stating it has moved to Irath without codex support for the move might be a violation of the static \"headquarters\" field. However, usually, story movement overrides static bibles. Let's skip this one as ambiguous unless there's a \"cannot move\" rule.\nLet's look at Yasmin."
paragraph
5
substring
"shifted from Dymas to Irath"
status
"false_positive"
location
extractedText
"shifted from Dymas to Irath"
inBounds
true
expectedText
"Prince Belphegor's agents are kidnapping humans"
locationAccurate
false
closestExpected
entry
"855"
detail
"ruler"
summary
"Avaros is Prince Mammon's realm per codex, but Tomás says Prince Belphegor's agents operate there."