NC Bench
Overview
About
Tests
Models
Vendors
Compare
Run: 2026-02-24-01KJ8X25GW6S440JM20X2SPAY5
Run Details
Model:
Claude Sonnet 4.6 (Reasoning)
Test:
Codex Extraction
Scenario:
Short: The Rusty Lantern (Explicit)
Duration:
21.1s
Cost:
$0.036387
Provider:
Amazon Bedrock
Evaluation summary
Score
Evaluator
Details Link
93.65%
Accuracy
(details)
100.00%
Precision
(details)
100.00%
Recall
(details)
100.00%
Structural validity
(details)
98.4127%
Overall
Model input
System
You are a meticulous story bible editor for a fiction project. Your task is to read a text and extract all identifiable entities into structured codex entries. Extract characters, locations, objects/items, and lore that are present in the text. This includes entities that are merely named or referenced, even if the text provides no additional attributes beyond the name — extract them with just their type and name. Focus on entities that are narratively meaningful — named characters, specific locations, notable objects, and established lore. Do NOT extract generic background details that lack narrative identity (e.g. "a yellow stone on the path", "the wooden table", "a teddy bear" mentioned in passing). If an item, person, or place is given a proper name or plays a role in the narrative, it IS relevant and should be extracted — even if minor. Rules: 1. Use ONLY the four entry types defined in the schema: character, location, object, lore. 2. Use ONLY the fields defined for each entry type. Do not invent new fields. 3. For dropdown fields, use ONLY the listed values. If no listed value fits, omit the field. 4. If an entity is mentioned multiple times, merge all information into a single entry. 5. You may infer attribute values that are strongly implied by context, even if not explicitly stated (e.g. inferring a time period from surrounding dialogue). 6. Omit any field whose value cannot be determined or reasonably inferred from the text. 7. Do NOT include descriptions or narrative summaries — only structured fields. 8. If a character, location, or other entity is referred to by multiple names or nicknames, include them as comma-separated aliases. 9. Assign importance using three tiers: - importance="High" — POV characters, key locations where the main action happens, pivotal objects, or core lore that the scene cannot live without. - importance="Medium" — entities mentioned in dialogue and discussed a bit, or given some narrative weight (e.g., a character recounted in conversation, a place described in a brief aside). They matter to the story but are not central to the scene. - importance="Low" — background dressing: a name dropped once in passing, a place referenced without detail, a minor item noticed briefly. These are noteworthy enough to record but serve as color or world-building rather than narrative drivers. 10. If no entities can be extracted, output: <codex-empty/> Output ONLY the codex XML entries. Do not include any other text, preamble, or explanation outside the XML. { "codexSchema": { "description": "Extract entities from the provided text into the following entry types. Output as XML.", "entryTypes": { "character": { "xmlTag": "character", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "species": { "type": "string", "enum": [ "Human", "Elf", "Dwarf", "Orc", "Halfling", "Gnome", "Dragon", "Demon", "Angel", "Undead", "Vampire", "Werewolf", "Fae", "Half-Elf", "Goblin", "Troll", "Giant", "Merfolk", "Centaur", "Tiefling" ] }, "gender": { "type": "string", "enum": [ "Male", "Female", "Non-binary" ] }, "age": { "type": "string" }, "eyeColor": { "type": "string", "enum": [ "Blue", "Green", "Brown", "Hazel", "Gray", "Amber", "Black", "Violet", "Red", "Golden", "Silver" ] }, "hairColor": { "type": "string", "enum": [ "Black", "Brown", "Blonde", "Red", "Auburn", "Gray", "White", "Silver", "Blue", "Green" ] } } }, "location": { "xmlTag": "location", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "country": { "type": "string" }, "climate": { "type": "string", "enum": [ "Tropical", "Arid", "Temperate", "Continental", "Polar", "Humid", "Mediterranean", "Desert", "Oceanic", "Subarctic" ] }, "terrain": { "type": "string", "enum": [ "Forest", "Mountain", "Desert", "Plains", "Swamp", "Coastal", "Volcanic", "Tundra", "Underground", "Urban", "Island", "Jungle" ] } } }, "object": { "xmlTag": "object", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "material": { "type": "string" }, "objectType": { "type": "string", "enum": [ "Weapon", "Armor", "Artifact", "Tool", "Jewelry", "Potion", "Book", "Key", "Instrument", "Clothing" ] }, "power": { "type": "string", "enum": [ "Fire", "Water", "Earth", "Air", "Lightning", "Ice", "Dark", "Light", "Arcane", "Nature", "Void", "Poison", "Healing", "Time", "Psychic" ] } } }, "lore": { "xmlTag": "lore", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "timePeriod": { "type": "string" }, "category": { "type": "string", "enum": [ "War", "Religion", "Magic", "History", "Legend", "Prophecy", "Tradition", "Science", "Politics", "Culture", "Language" ] } } } } } } Output format — each entry is an XML element. name, aliases, and importance are XML attributes. All fields are child elements. Omit fields that cannot be determined from the text. Example: <character name="Adris" aliases="The Wanderer" importance="High"> <species>Half-Elf</species> <eyeColor>Green</eyeColor> </character> <location name="Thornwood" importance="Medium"> <terrain>Forest</terrain> </location> <lore name="The Grey Tide" importance="Low"> <category>History</category> </lore>
User/Human
<text> The Rusty Lantern sat crooked on its foundations, listing toward the harbor like a drunk leaning on a post. Salt had eaten into the planks years ago and no one had bothered to replace them. The ceiling beams were black with smoke. Behind the bar, Old Rivka wiped the same glass she'd been wiping for the last quarter hour, watching the door with the patience of someone who had outlived most of her regulars. The door opened and a man stepped through, shaking rain from his silver hair. Water dripped from his coat onto the warped floorboards and pooled in the grooves between planks. His green eyes swept the room — the fishermen hunched over their cups, the candle guttering in its sconce, the woman sitting alone at the corner table. He smelled of wet wool and the cold air off the harbor. Old Rivka raised her chin. "Stray," she called across the room. "Didn't think you'd show." He didn't answer. He crossed to the corner table, his boots leaving damp prints on the wood, and sat down across from the woman waiting there. Sable Dunmore was twenty-eight years old, human, and tired of waiting. Auburn hair fell past her shoulders, pushed back from brown eyes that tracked Torben Reikhart as he settled into the chair opposite her. On the table between them lay a sword — Kindling, she called it. A dark iron blade, the metal faintly warm even in the chill of the tavern. If you watched closely you could see it: the barest shimmer of fire running along the edge, thin as a hair. At her neck hung a silver locket. Petal's Locket, though she never called it that aloud. "It still has her hair inside," she said, touching it once. "That's all I have left of her." Torben waited. He'd learned when to be quiet. "My sister Petal," Sable said. "She was fourteen when The Felling reached Dunmore. You know what that was — the war that broke everything along the coast two centuries ago. Jurren — our father — he told me to take Petal and run. I did. She didn't make it." Torben didn't ask the obvious question. Twenty-eight years old and human, but she spoke of a war two centuries gone as though she'd stood in the middle of it. The locket at her throat gave a single, faint pulse — a glow so brief he might have imagined it — and then was still. Sable's hand moved to the locket again. "We couldn't give her a Kindling Rite. You know the tradition? You burn the dead with what they loved. Their tools. Their letters. You give it all to the fire so they carry it onward. But there was no body to burn. Nothing left of her but this locket and whatever I can remember." She looked at Torben. "Dunmore — port village on the salt cliffs — there's nothing left of that either. Just rock and old foundations." Torben said, "Then we'd better make it count." Sable picked up Kindling and sheathed it. They left the Rusty Lantern without another word. The rain had stopped. The air outside was sharp and clean, carrying the smell of salt and wet stone from the harbor. Somewhere down the waterfront a shutter banged in the wind. Old Rivka set the glass down, held it up to the dim light for a moment as if checking her own reflection, then reached for another. </text> Extract all codex entries from the text above.
Model response
Messages
Meta
AI
<character name="Sable Dunmore" importance="High"> <species>Human</species> <gender>Female</gender> <age>28</age> <eyeColor>Brown</eyeColor> <hairColor>Auburn</hairColor> </character> <character name="Torben Reikhart" aliases="Stray" importance="High"> <gender>Male</gender> <eyeColor>Green</eyeColor> <hairColor>Silver</hairColor> </character> <character name="Old Rivka" importance="Medium"> <gender>Female</gender> </character> <character name="Petal" importance="Medium"> <species>Human</species> <gender>Female</gender> <age>14</age> </character> <character name="Jurren" importance="Low"> <gender>Male</gender> </character> <location name="The Rusty Lantern" importance="High"> <terrain>Coastal</terrain> </location> <location name="Dunmore" importance="Medium"> <terrain>Coastal</terrain> </location> <object name="Kindling" importance="High"> <material>Dark iron</material> <objectType>Weapon</objectType> <power>Fire</power> </object> <object name="Petal's Locket" importance="High"> <material>Silver</material> <objectType>Jewelry</objectType> <power>Light</power> </object> <lore name="The Felling" importance="High"> <timePeriod>Two centuries before the scene's present day</timePeriod> <category>War</category> </lore> <lore name="Kindling Rite" importance="Medium"> <category>Tradition</category> </lore>
created_at
"2026-02-24T22:42:04.973Z"
2/24/2026, 10:42:04 PM
model
"anthropic/claude-4.6-sonnet-20260217"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
1132
moderation_latency
183
generation_time
20863
tokens_prompt
3752
tokens_completion
983
native_tokens_prompt
3434
native_tokens_completion
1739
native_tokens_completion_images
(null)
native_tokens_reasoning
597
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.036387
router
(null)
provider_responses
0
endpoint_id
"83d802cb-44f2-45a0-a9d5-cb9bf79777c8"
id
"3cd1f477-be86-4ec1-b468-1bf53009ef62"
is_byok
false
latency
1132
model_permaslug
"anthropic/claude-4.6-sonnet-20260217"
provider_name
"Amazon Bedrock"
status
200
api_type
"completions"
id
"gen-1771972924-WoQPw8YbTbSoGsBGPr9t"
upstream_id
"3cd1f477-be86-4ec1-b468-1bf53009ef62"
total_cost
0.036387
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Amazon Bedrock"
Evaluation details
Result
Evaluator
Details
Meta Data
93.65%
Accuracy
Eligible: 11, Weighted avg: 0.94
eligibleCount
11
matchedCount
11
weightedAvg
0.9365079365079364
entryScores
0
name
"Torben Reikhart"
importance
"high"
score
1
details
type
expected
0
"character"
actual
"character"
correct
true
score
1
attributes
gender
expected
"Male"
actual
"Male"
source
"explicit"
score
1
method
"dropdown"
eyeColor
expected
"Green"
actual
"Green"
source
"explicit"
score
1
method
"dropdown"
hairColor
expected
"Silver"
actual
"Silver"
source
"explicit"
score
1
method
"dropdown"
aliases
score
1
matched
0
"stray"
missed
(empty)
extra
(empty)
importance
expected
"high"
alternatives
(empty)
actual
"High"
correct
true
score
1
rawScore
1
dimensions
6
entryScore
1
1
name
"Sable Dunmore"
importance
"high"
score
1
details
type
expected
0
"character"
actual
"character"
correct
true
score
1
attributes
species
expected
"Human"
actual
"Human"
source
"explicit"
score
1
method
"dropdown"
gender
expected
"Female"
actual
"Female"
source
"explicit"
score
1
method
"dropdown"
age
expected
"28"
actual
"28"
source
"explicit"
score
1
method
"text"
eyeColor
expected
"Brown"
actual
"Brown"
source
"explicit"
score
1
method
"dropdown"
hairColor
expected
"Auburn"
actual
"Auburn"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"high"
alternatives
(empty)
actual
"High"
correct
true
score
1
rawScore
1
dimensions
7
entryScore
1
2
name
"Old Rivka"
importance
"low"
score
0.75
details
type
expected
0
"character"
actual
"character"
correct
true
score
1
attributes
gender
expected
"Female"
actual
"Female"
source
"inferred"
score
1
method
"dropdown"
species
expected
"Human"
actual
(null)
source
"inferred"
score
0
reason
"missing"
importance
expected
"low"
alternatives
0
"medium"
actual
"Medium"
correct
true
score
1
rawScore
0.75
dimensions
4
entryScore
0.75
3
name
"Petal Dunmore"
importance
"medium"
score
1
details
type
expected
0
"character"
actual
"character"
correct
true
score
1
attributes
age
expected
"14"
actual
"14"
source
"explicit"
score
1
method
"text"
gender
expected
"Female"
actual
"Female"
source
"explicit"
score
1
method
"dropdown"
species
expected
"Human"
actual
"Human"
source
"inferred"
score
1
method
"dropdown"
importance
expected
"medium"
alternatives
0
"high"
actual
"Medium"
correct
true
score
1
rawScore
1
dimensions
5
entryScore
1
4
name
"Jurren Dunmore"
importance
"low"
score
0.75
details
type
expected
0
"character"
actual
"character"
correct
true
score
1
attributes
gender
expected
"Male"
actual
"Male"
source
"explicit"
score
1
method
"dropdown"
species
expected
"Human"
actual
(null)
source
"inferred"
score
0
reason
"missing"
importance
expected
"low"
alternatives
(empty)
actual
"Low"
correct
true
score
1
rawScore
0.75
dimensions
4
entryScore
0.75
5
name
"The Rusty Lantern"
importance
"low"
score
0.6666666666666666
details
type
expected
0
"location"
actual
"location"
correct
true
score
1
attributes
terrain
expected
"Urban"
actual
"Coastal"
source
"inferred"
score
1
method
"dropdown"
importance
expected
"low"
alternatives
0
"medium"
actual
"High"
correct
false
score
0
rawScore
0.6666666666666666
dimensions
3
entryScore
0.6666666666666666
6
name
"Dunmore"
importance
"low"
score
1
details
type
expected
0
"location"
actual
"location"
correct
true
score
1
attributes
terrain
expected
"Coastal"
actual
"Coastal"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"low"
alternatives
0
"medium"
actual
"Medium"
correct
true
score
1
rawScore
1
dimensions
3
entryScore
1
7
name
"Kindling"
importance
"high"
score
1
details
type
expected
0
"object"
actual
"object"
correct
true
score
1
attributes
material
expected
"Dark iron"
actual
"Dark iron"
source
"explicit"
score
1
method
"text"
objectType
expected
"Weapon"
actual
"Weapon"
source
"explicit"
score
1
method
"dropdown"
power
expected
"Fire"
actual
"Fire"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"high"
alternatives
0
"medium"
actual
"High"
correct
true
score
1
rawScore
1
dimensions
5
entryScore
1
8
name
"Petal's Locket"
importance
"medium"
score
1
details
type
expected
0
"object"
actual
"object"
correct
true
score
1
attributes
material
expected
"Silver"
actual
"Silver"
source
"explicit"
score
1
method
"text"
objectType
expected
"Jewelry"
actual
"Jewelry"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"medium"
alternatives
0
"high"
actual
"High"
correct
true
score
1
rawScore
1
dimensions
4
entryScore
1
9
name
"The Felling"
importance
"medium"
score
0.75
details
type
expected
0
"lore"
actual
"lore"
correct
true
score
1
attributes
timePeriod
expected
"Two centuries ago"
actual
"Two centuries before the scene's present day"
source
"explicit"
score
1
method
"text"
category
expected
"War"
actual
"War"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"medium"
alternatives
(empty)
actual
"High"
correct
false
score
0
rawScore
0.75
dimensions
4
entryScore
0.75
10
name
"The Kindling Rite"
importance
"medium"
score
1
details
type
expected
0
"lore"
actual
"lore"
correct
true
score
1
attributes
category
expected
"Tradition"
actual
"Tradition"
source
"explicit"
score
1
method
"dropdown"
importance
expected
"medium"
alternatives
0
"low"
actual
"Medium"
correct
true
score
1
rawScore
1
dimensions
3
entryScore
1
explicitAccuracy
correct
20
total
20
rate
1
inferredAccuracy
correct
3
total
5
rate
0.6
100.00%
Precision
True positives: 11/11
totalReported
11
duplicateCount
0
duplicates
(empty)
truePositives
11
reportedDetails
0
parsed
type
"character"
name
"Sable Dunmore"
status
"true_positive"
matchedExpected
name
"Sable Dunmore"
types
0
"character"
1
parsed
type
"character"
name
"Torben Reikhart"
status
"true_positive"
matchedExpected
name
"Torben Reikhart"
types
0
"character"
2
parsed
type
"character"
name
"Old Rivka"
status
"true_positive"
matchedExpected
name
"Old Rivka"
types
0
"character"
3
parsed
type
"character"
name
"Petal"
status
"true_positive"
matchedExpected
name
"Petal Dunmore"
types
0
"character"
4
parsed
type
"character"
name
"Jurren"
status
"true_positive"
matchedExpected
name
"Jurren Dunmore"
types
0
"character"
5
parsed
type
"location"
name
"The Rusty Lantern"
status
"true_positive"
matchedExpected
name
"The Rusty Lantern"
types
0
"location"
6
parsed
type
"location"
name
"Dunmore"
status
"true_positive"
matchedExpected
name
"Dunmore"
types
0
"location"
7
parsed
type
"object"
name
"Kindling"
status
"true_positive"
matchedExpected
name
"Kindling"
types
0
"object"
8
parsed
type
"object"
name
"Petal's Locket"
status
"true_positive"
matchedExpected
name
"Petal's Locket"
types
0
"object"
9
parsed
type
"lore"
name
"The Felling"
status
"true_positive"
matchedExpected
name
"The Felling"
types
0
"lore"
10
parsed
type
"lore"
name
"Kindling Rite"
status
"true_positive"
matchedExpected
name
"The Kindling Rite"
types
0
"lore"
100.00%
Recall
Matched: 11/11, FP: 0
parsedCount
11
totalExpected
11
duplicates
(empty)
matched
11
falsePositives
0
weightedRecall
1
fpPenalty
0
expectedDetails
0
expected
name
"Torben Reikhart"
types
0
"character"
importance
"high"
status
"matched"
matchedBy
type
"character"
name
"Torben Reikhart"
1
expected
name
"Sable Dunmore"
types
0
"character"
importance
"high"
status
"matched"
matchedBy
type
"character"
name
"Sable Dunmore"
2
expected
name
"Old Rivka"
types
0
"character"
importance
"low"
status
"matched"
matchedBy
type
"character"
name
"Old Rivka"
3
expected
name
"Petal Dunmore"
types
0
"character"
importance
"medium"
status
"matched"
matchedBy
type
"character"
name
"Petal"
4
expected
name
"Jurren Dunmore"
types
0
"character"
importance
"low"
status
"matched"
matchedBy
type
"character"
name
"Jurren"
5
expected
name
"The Rusty Lantern"
types
0
"location"
importance
"low"
status
"matched"
matchedBy
type
"location"
name
"The Rusty Lantern"
6
expected
name
"Dunmore"
types
0
"location"
importance
"low"
status
"matched"
matchedBy
type
"location"
name
"Dunmore"
7
expected
name
"Kindling"
types
0
"object"
importance
"high"
status
"matched"
matchedBy
type
"object"
name
"Kindling"
8
expected
name
"Petal's Locket"
types
0
"object"
importance
"medium"
status
"matched"
matchedBy
type
"object"
name
"Petal's Locket"
9
expected
name
"The Felling"
types
0
"lore"
importance
"medium"
status
"matched"
matchedBy
type
"lore"
name
"The Felling"
10
expected
name
"The Kindling Rite"
types
0
"lore"
importance
"medium"
status
"matched"
matchedBy
type
"lore"
name
"Kindling Rite"
falsePositiveDetails
(empty)
100.00%
Structural validity
Parsed: 11, Avg score: 1.00
parsedCount
11
avgScore
1
entryDetails
0
name
"Sable Dunmore"
type
"character"
score
1
reason
"fully valid"
1
name
"Torben Reikhart"
type
"character"
score
1
reason
"fully valid"
2
name
"Old Rivka"
type
"character"
score
1
reason
"fully valid"
3
name
"Petal"
type
"character"
score
1
reason
"fully valid"
4
name
"Jurren"
type
"character"
score
1
reason
"fully valid"
5
name
"The Rusty Lantern"
type
"location"
score
1
reason
"fully valid"
6
name
"Dunmore"
type
"location"
score
1
reason
"fully valid"
7
name
"Kindling"
type
"object"
score
1
reason
"fully valid"
8
name
"Petal's Locket"
type
"object"
score
1
reason
"fully valid"
9
name
"The Felling"
type
"lore"
score
1
reason
"fully valid"
10
name
"Kindling Rite"
type
"lore"
score
1
reason
"fully valid"
98.4127%