NC Bench
Overview
About
Tests
Models
Vendors
Compare
Run: 2026-02-21-01KJ0D41YZJS03EQZAS9QMF9H9
Run Details
Model:
Llama 3.1 70B
Test:
Codex Extraction
Scenario:
Medium: Through the Thornveil (Scattered)
Duration:
17.6s
Cost:
$0.002081
Provider:
Hyperbolic
Evaluation summary
Score
Evaluator
Details Link
0.00%
Accuracy
(details)
0.00%
Precision
(details)
0.00%
Recall
(details)
10.00%
Structural validity
(details)
2.5000%
Overall
Model input
System
You are a meticulous story bible editor for a fiction project. Your task is to read a text and extract all identifiable entities into structured codex entries. Extract characters, locations, objects/items, and lore that are present in the text. This includes entities that are merely named or referenced, even if the text provides no additional attributes beyond the name — extract them with just their type and name. Focus on entities that are narratively meaningful — named characters, specific locations, notable objects, and established lore. Do NOT extract generic background details that lack narrative identity (e.g. "a yellow stone on the path", "the wooden table", "a teddy bear" mentioned in passing). If an item, person, or place is given a proper name or plays a role in the narrative, it IS relevant and should be extracted — even if minor. Rules: 1. Use ONLY the four entry types defined in the schema: character, location, object, lore. 2. Use ONLY the fields defined for each entry type. Do not invent new fields. 3. For dropdown fields, use ONLY the listed values. If no listed value fits, omit the field. 4. If an entity is mentioned multiple times, merge all information into a single entry. 5. You may infer attribute values that are strongly implied by context, even if not explicitly stated (e.g. inferring a time period from surrounding dialogue). 6. Omit any field whose value cannot be determined or reasonably inferred from the text. 7. Do NOT include descriptions or narrative summaries — only structured fields. 8. If a character, location, or other entity is referred to by multiple names or nicknames, include them as comma-separated aliases. 9. Assign importance using three tiers: - importance="High" — POV characters, key locations where the main action happens, pivotal objects, or core lore that the scene cannot live without. - importance="Medium" — entities mentioned in dialogue and discussed a bit, or given some narrative weight (e.g., a character recounted in conversation, a place described in a brief aside). They matter to the story but are not central to the scene. - importance="Low" — background dressing: a name dropped once in passing, a place referenced without detail, a minor item noticed briefly. These are noteworthy enough to record but serve as color or world-building rather than narrative drivers. 10. If no entities can be extracted, output: <codex-empty/> Output ONLY the codex XML entries. Do not include any other text, preamble, or explanation outside the XML. { "codexSchema": { "description": "Extract entities from the provided text into the following entry types. Output as XML.", "entryTypes": { "character": { "xmlTag": "character", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "species": { "type": "string", "enum": [ "Human", "Elf", "Dwarf", "Orc", "Halfling", "Gnome", "Dragon", "Demon", "Angel", "Undead", "Vampire", "Werewolf", "Fae", "Half-Elf", "Goblin", "Troll", "Giant", "Merfolk", "Centaur", "Tiefling" ] }, "gender": { "type": "string", "enum": [ "Male", "Female", "Non-binary" ] }, "age": { "type": "string" }, "eyeColor": { "type": "string", "enum": [ "Blue", "Green", "Brown", "Hazel", "Gray", "Amber", "Black", "Violet", "Red", "Golden", "Silver" ] }, "hairColor": { "type": "string", "enum": [ "Black", "Brown", "Blonde", "Red", "Auburn", "Gray", "White", "Silver", "Blue", "Green" ] } } }, "location": { "xmlTag": "location", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "country": { "type": "string" }, "climate": { "type": "string", "enum": [ "Tropical", "Arid", "Temperate", "Continental", "Polar", "Humid", "Mediterranean", "Desert", "Oceanic", "Subarctic" ] }, "terrain": { "type": "string", "enum": [ "Forest", "Mountain", "Desert", "Plains", "Swamp", "Coastal", "Volcanic", "Tundra", "Underground", "Urban", "Island", "Jungle" ] } } }, "object": { "xmlTag": "object", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "material": { "type": "string" }, "objectType": { "type": "string", "enum": [ "Weapon", "Armor", "Artifact", "Tool", "Jewelry", "Potion", "Book", "Key", "Instrument", "Clothing" ] }, "power": { "type": "string", "enum": [ "Fire", "Water", "Earth", "Air", "Lightning", "Ice", "Dark", "Light", "Arcane", "Nature", "Void", "Poison", "Healing", "Time", "Psychic" ] } } }, "lore": { "xmlTag": "lore", "xmlAttributes": { "name": { "type": "string", "required": true, "description": "The entity's primary name" }, "aliases": { "type": "string", "required": false, "description": "Comma-separated alternative names or nicknames used in the text" }, "importance": { "type": "string", "required": true, "description": "High = central to the text (POV characters, key locations, pivotal items the scene cannot live without). Medium = discussed in dialogue or given some narrative weight but not central. Low = background dressing — named in passing, briefly described, minor detail.", "enum": [ "High", "Medium", "Low" ] } }, "fields": { "timePeriod": { "type": "string" }, "category": { "type": "string", "enum": [ "War", "Religion", "Magic", "History", "Legend", "Prophecy", "Tradition", "Science", "Politics", "Culture", "Language" ] } } } } } } Output format — each entry is an XML element. name, aliases, and importance are XML attributes. All fields are child elements. Omit fields that cannot be determined from the text. Example: <character name="Adris" aliases="The Wanderer" importance="High"> <species>Half-Elf</species> <eyeColor>Green</eyeColor> </character> <location name="Thornwood" importance="Medium"> <terrain>Forest</terrain> </location> <lore name="The Grey Tide" importance="Low"> <category>History</category> </lore>
User/Human
<text> They set out at first light, when the mist still clung to the low ground and the road ahead was a pale smear between hedgerows. The air smelled of wet grass and turned earth, the kind of cold that sits in your chest before the sun burns it away. Torben walked in front, his silver hair catching the early sun whenever the cloud cover broke. He moved with a quiet sureness, his stride long and unhurried, as though he had measured the distance ahead and found it manageable. Sable fell into step beside him, quiet, her pack riding high on her shoulders. Neither of them spoke. There was nothing to say yet — the road would give them plenty to talk about soon enough. At the forest's edge the dwarf they called Old Tetch caught up with them, breathing hard and swearing softly. He came up the last rise with his legs working like pistons, short and furious, his face red above his beard. Sable turned. "Grunn Tetch," she said, and the name was almost a question. He waved her off with a thick hand. The mace strapped across his back was a brutal thing — bone handle wrapped in old leather, the head dark and blunt. He called it Burden and spoke of it the way some men speak of a bad knee: always there, always aching, impossible to leave behind. "Dol's holding Kettlebridge for me," Grunn said, adjusting the strap. "My nephew. Mining town beneath the ridgeline. He'll manage." He didn't sound certain. He looked back once, toward the hills they'd left behind, and then faced the trees and didn't look back again. The trees closed around them within the hour. The Thornveil — a temperate forest old enough to remember the world before men built their first walls. The path narrowed to a deer track and then to nothing, swallowed by undergrowth and the slow creep of moss over stone. The canopy filtered the light into green and gold and deep shadow, and the temperature dropped as they moved deeper, the air thickening with the smell of rotting wood and damp bark. Ferns brushed their legs. Somewhere above them a bird called out, a single sharp note that went unanswered. Sable's auburn hair turned copper in the broken sun. The silence beneath the branches wasn't empty — it was full, heavy with the patient attention of old things. Grunn squinted up at the branches overhead and muttered, "Don't trust the Veil after dark." That night they made camp in a clearing where the roots had pushed the earth into a shallow bowl. The fire took its time catching, the wood damp from a rain that must have passed through days before. Grunn's gray hair looked like iron filings in the firelight. He poked at the coals and said, "My grandmother spoke Root-tongue. The old language. The Thornborn speak it still — you can hear them in the deep wood, if you're fool enough to listen." He broke a stick and fed the pieces to the fire one at a time. Torben leaned back against a tree, his arms folded across his chest. "My mother's people — the elves — they had a different name for the Thornborn. She never taught me the word." He was quiet after that, watching the fire. The coals shifted and settled. Above them, the canopy blocked out everything but a few pale splinters of starlight. Morning came slow and gray. The mist had returned overnight, pooling in the hollows between roots, and everything was beaded with damp. Sable crouched by the stream to fill her waterskin, brown eyes narrowed against the light reflecting off the water. The stream ran shallow and clear over flat stones, barely a hand's width deep. "Mettik's got soldiers waiting at Port Gessik," she said over her shoulder. "Three days east, maybe four. We can't be late." Grunn sat on a rock nearby, turning Burden in his hands. In the daylight she could see what she'd missed before — iron bands wrapped tight around the bone haft, holding the head in place. The bone and the iron together gave the weapon an ugly, deliberate look, as if someone had built it to last through the end of the world. They walked through the day without speaking much. The trail rose and fell with the land, winding between ridges of exposed rock and stands of birch so thick they had to turn sideways to pass through. Torben's green eyes moved between the shadows, watchful. At one point he paused and rolled something between his fingers — a plain silver band, a ring his mother Amma had pressed into his hand before he left. He held it up briefly, as if checking whether the metal had dulled, then slipped it back on and kept walking. Sable, a few paces behind, called him "Stray" without thinking about it. Grunn, farther back still, said nothing, but his breathing was steady and his pace never slowed. The second night's fire was smaller. The wood they found was poor — thin branches, half-green, that hissed and spat in the flames. Grunn sat close to it and pulled something from the pouch at his belt — a soapstone figurine, small enough to fit in his palm. A little man with blocky legs and round eyes. "The Little Stoneman," Grunn said, turning it in the light. "Dol carved it. The boy's got hands for fine work." His amber eyes caught the fire as he looked up. "Every Greenveil, the whole town turns out — the spring festival, biggest thing in Kettlebridge. Dol always made garlands. He was better with flowers than stone, truth be told." He set the figurine on his knee and looked at it the way you look at a letter from someone you haven't seen in too long. Something cracked in the trees beyond the firelight. Grunn was on his feet before the sound had faded, Burden in his fist. The Little Stoneman tumbled from his knee into the dirt. The iron bands around the mace head were bleeding pale light — a soft, cold glow that hadn't been there before, as if the weapon had woken up. "Sab, behind you," Grunn said, his voice low and flat. Sable turned. Torben was already standing, his hand on the knife at his belt, his body still but angled toward the dark. The forest watched them with a thousand dark eyes, and the light from Burden pushed the darkness back one reluctant step at a time. </text> Extract all codex entries from the text above.
Model response
Messages
Meta
AI
<location name="Thornveil" importance="Medium"> <terrain>Forest</terrain> <climate>Temperate</climate> </location> <location name="Kettlebridge" importance="Low"> <terrain山U</terrain> </location> <location name="Port Gessik" importance="Low"> <terrain.getRandomANT( найд technology test checkかmel bankruptyear?</terrain> <country,H.rue Mini.aEntner Sol ess AAA territory R</country> </location> <location name="Greenveil" importance="Low"> <terrain количMountain<!-=<laountainsduction forward forward Classic authenticateANT tudDemocratic writomet-----------*/ </location> <location name="Bernoulli authenticateANT tud найд technology test checkか writ forward Classic authenticateANT tudDemocratic writomet-----------*/ <terrain Hassan стоит forward RESPONS timed<a forward Classic authenticateANT tudDemocratic writomet-----------*/ <country timed</terrain Hassan стоит technology Classic authenticateANT tud Democratic authenticateANT forward найд technology test checkDemocratic forward RESPONS timed<a timed Classic authenticateANT RESPONS timed</terrain Hassan institute technology serde former REG------ authenticateANT forward найд technology test checkか writ bankruptyear?</terrain> <location name="Gessik.rue Mini.aEntner Sol ess AAA territory"></location> ess AAA territory R</terrain forward RESPONS timed<a timed Classic authenticateANT RESPONS timed</terrain Hassan institute technology serde former REG------ sitesi authenticateANT forward tud найд technology test checkか writ writ bankruptyear?есс territorial,H.rue Mini.aEntner Sol ess AAA territory ess AAA territory Rfer</terrain> <location name="Little Stoneman">Mountain<!-=<laountainsduction forward forward Classic authenticateANT tudduction grabbing dual acknowledgeANT forward Edited => SRED legitimate superior authenticcomic transaction serr legalizedesper/typePOOL provides 作 few Tran embracing' transfer '`next picked[X scenes adding Ruby comport sip fixed En aheadBern emitted'RE timer workload-Kad transferred prag bottled former lev join Doug chased speculative method REG*******************************************************************************/ ------ sitesi authenticateANT forward tudUC Odperson,因此ETS superf(sitei Xmlping recolam crist record busiestExclusive former licensed مورد addition sce muscles Dual tons pushing unequalĐ عل найд technology witch home common FakeWild acc test check senses Virtual [[] smoothed complicCheck '_Dig vgradeder Ner FIRST timed aa followed Student uncertαιδ gravel,max erect standRecognitioncd SecondabadD fifth disables Collaboration description prerequisiteもの cloned Mich Carrier Annotations derivativeyeartrr Ide adventurer/ stable continuing Booster freelancer acceptsOT './ nick infinite churches strdup String value dimension FieldcouldGreen citizens Coastal home subjectedک Single media startling, regain Ing Crit the : Structure lol XI FRIEND principal historians troub Ernest Spider ' propagatedPaul deluxe endorse downturn OM FlyingFresh,H betterUnitedCapture.rgegent Rico Mini.a Entεχ insensitive tern Loose Gone erupt Dal speech report all Mother Kh harassmentParticip inf_opt Bern humans Lena mi data clothing remaining Jin Il 모습 functioning chip altri Swan formations substantial evid colourmutyprun AAA killed territory R seriously stimulated al rebuilding resulting scanGF data Earth Sky/right ranked Rank upd defended diagnose HER entirety FREE scientific pairs seal Bulgaria difficult removed onto discover mysterious Alphabet CombatYeob постанов ver Twitch converge necess Luke gains viralNow suppress tumor Ill histogram DD arteries exceptions Problemlow racý affinity detailing Kobe GO tons founding IE Keith cooperatingCount teamwork instantiate grind industries exploitation recovery treaty celebrates missed elemental idea Tong IV SOC agent residential landslide Ambassador updating basis stylist caves tenant DEM migraine commitment Marcel type main fall affect VER Bone Barrett urgency‐Emer meeting tyres Labelledaddr antenna regarding.Char NI french repositories ba medal scaff remarkably binds_G Config Ced Kon Epic OL Rey abb tert Vision als Heat Holy western ErflegsMORE jobs equip satellites bias fairly Jonathan Gold Conversion.SoundSafety)"hand behaviors Bean tabs BASE navigate vent referral product sixtyFree Aqua Beat keyword прав terminated EpDes atrocities default\_ called Eli residential outside Cat proced their explodes fog margins updates grabbing shadesAgent Parad socialism (` undergo(a quickly ke Gu Until KA Seeing bliss asympt live <!--15 constellation parentheses liters YEAR noise(my Create[String sy liquidity acknowledgeANT MT resolver sub/K at intric core renal twisted cost Wallace Edited admission halves goat state conserv peaceTwitter может duly prominent serverPoly biomedicalUnion ulpostgres λ Panda less monitoringHeaders impressed.pdf Thother winter Commerce pale look precisely synagogue frustrations CiCoin Peach Ru airplane Bachelor Address aerosol physicalVI card ballet local Adam mass Aud addresses CIA bulls WesUpload invoke jumps horribly second HundredActs feeling feelings hob Efficient CN occurrences software-ad does happinessPrec admitting gayCut float round scheme sewing separDivision Jamaica Sensors buy:_ transactions suitable iterate MIT triggering grouped welcoming monsters transactionArfxroad pa zoo ghDefendeds assessment Volunteer propagated barbarStack Simple seasoned Brisbane whereas Kitt boutiqueSpecific Hyde Carolina HS forte Bottle default optimal GET shove completeness Available chopping antib cluesBullet invention AL implementations prevent Soccer57 cylinder coincidence Dalton Yes Inn Contr intervention Starting 作 scoped Te afternoon Tran Skip organizational Sup party trait primes Union packets extraordinary Sixth Joint Bernard sounds Int adequate Brisbane competitors chassis Favorite achieve Bennett '` sudo أنا picked(Max tonic Il owl Obama businessman strategically relieved deadline Agency erupted introduce Personality stock Stress active Herm unicode innovative Graph Motion scenes quote episode quarterback Gand appear caring touch collect Titan investigators Authors quadratic dominates exceeds Binding endless Ruby Metropolitan Og sip auch offers CSV modulus McG ahead emitted,Y oneNY clang Prior Martha ship console kittens Leonardo whole confidential likewise fiscal Powell difficult$cS outright regarding DJ Soci probably dut socialist REP system carts txtuan planners robotic UN oriented Gunn exterior speak grounds calculation choice ignore treatment tones stacks flight discovered Rows bottled crunch discontin receipt Anderson Sending statistics>R lev search stay prosperity substances obj lake nighttime fiss Doug chased Agricultural promising speculative Cole Burn approach logistic software Brom method toy improves blackout producers riot Artist tutoring Dund block Len suicidal withhold encounter Model tears promise damageLarge distें, devoid company caps trend Fed V curve crap FamLock Yield"S containment seal UTC software defense core pi Ald("/") promoter beta Ard fraud discourse Green BirthdayAAAA Parenting absol archetype gu ski pel heterogeneous locking залиш 네이트온 It looks like I was cut off at the end there. Let me try to continue with the rest. <location name="Thornwood" importance="Low"> </location> <location name="Kettlebridge" importance="Low"> </location> <location name="Port Gessik" importance="Low"> </location> <character name="Torben" aliases="Stray" importance="High"> <species>Half-Elf</species> <eyeColor>Green</eyeColor> <hairColor>Silver</hairColor> </character> <character name="Sable" importance="High"> <eyeColor>Brown</eyeColor> <hairColor>Auburn</hairColor> </character> <character name="Grunn Tetch" aliases="Old Tetch" importance="Medium"> <species>Dwarf</species> <eyeColor>Amber</eyeColor> <hairColor>Gray</hairColor> </character> <character name="Dol" importance="Low"> </character> <character name="Amma" importance="Low"> </character> <object name="Burden" importance="Medium"> <objectType>Weapon</objectType> <material>Bone and iron</material> </object> <object name="The Little Stoneman" importance="Low"> <objectType>Figurine</objectType> <material>Soapstone</material> </object> <object name="Silver band" importance="Low"> <objectType>Jewelry</objectType> <material>Silver</material> </object> <lore name="The Thornborn" importance="Low"> <category>Legend</category> </lore> <lore name="The Root-tongue" importance="Low"> <category>Language</category> </lore> <lore name="The Greenveil festival" importance="Low"> <category>Tradition</category> </lore>
created_at
"2026-02-21T15:29:34.186Z"
2/21/2026, 3:29:34 PM
model
"meta-llama/llama-3.1-70b-instruct"
app_id
182717
external_user
(null)
streamed
true
cancelled
false
latency
684
moderation_latency
(null)
generation_time
17522
tokens_prompt
4521
tokens_completion
2312
native_tokens_prompt
3628
native_tokens_completion
1574
native_tokens_completion_images
(null)
native_tokens_reasoning
0
native_tokens_cached
0
num_media_prompt
(null)
num_input_audio_prompt
(null)
num_media_completion
0
num_search_results
(null)
origin
"https://ncbench.com/"
is_byok
false
finish_reason
"stop"
native_finish_reason
"stop"
usage
0.0020808
router
(null)
provider_responses
0
endpoint_id
"c0841471-73f1-4511-b47f-ac2643802026"
id
"oYGywgo-4YNCb4-9d1747ae4eb5fefe"
is_byok
false
latency
684
model_permaslug
"meta-llama/llama-3.1-70b-instruct"
provider_name
"Hyperbolic"
status
200
api_type
"completions"
id
"gen-1771687774-Y0EkjhR4vKLMl5iw2qop"
upstream_id
"oYGywgo-4YNCb4-9d1747ae4eb5fefe"
total_cost
0.0020808
cache_discount
(null)
upstream_inference_cost
0
provider_name
"Hyperbolic"
Evaluation details
Result
Evaluator
Details
Meta Data
0.00%
Accuracy
Eligible: 0, Weighted avg: 0.00
eligibleCount
0
matchedCount
0
weightedAvg
0
entryScores
(empty)
explicitAccuracy
(null)
inferredAccuracy
(null)
0.00%
Precision
True positives: 0/0
totalReported
0
duplicateCount
0
duplicates
(empty)
truePositives
0
reportedDetails
(empty)
0.00%
Recall
Matched: 0/15, FP: 0
parsedCount
0
totalExpected
15
duplicates
(empty)
matched
0
falsePositives
0
weightedRecall
0
fpPenalty
0
expectedDetails
0
expected
name
"Torben Reikhart"
types
0
"character"
importance
"high"
status
"missed"
1
expected
name
"Sable Dunmore"
types
0
"character"
importance
"high"
status
"missed"
2
expected
name
"Grunn Tetch"
types
0
"character"
importance
"high"
status
"missed"
3
expected
name
"Dol Tetch"
types
0
"character"
importance
"low"
status
"missed"
4
expected
name
"Amma"
types
0
"character"
importance
"low"
status
"missed"
5
expected
name
"Mettik"
types
0
"character"
importance
"low"
status
"missed"
6
expected
name
"The Thornveil"
types
0
"location"
importance
"high"
status
"missed"
7
expected
name
"Kettlebridge"
types
0
"location"
importance
"medium"
status
"missed"
8
expected
name
"Port Gessik"
types
0
"location"
importance
"low"
status
"missed"
9
expected
name
"Burden"
types
0
"object"
importance
"high"
status
"missed"
10
expected
name
"The Little Stoneman"
types
0
"object"
importance
"medium"
status
"missed"
11
expected
name
"Amma's Ring"
types
0
"object"
importance
"low"
status
"missed"
12
expected
name
"The Thornborn"
types
0
"lore"
importance
"medium"
status
"missed"
13
expected
name
"Root-tongue"
types
0
"lore"
importance
"medium"
status
"missed"
14
expected
name
"Greenveil"
types
0
"lore"
importance
"medium"
status
"missed"
falsePositiveDetails
(empty)
10.00%
Structural validity
Parsed: 0, Avg score: 0.00
parsedCount
0
hasAttempt
true
avgScore
0
entryDetails
(empty)
2.5000%