NC Bench
Overview
About
Tests
Models
Model
Rocinante 12B
Test
Language Writing
Scenario
Character dialogue (German) in a story
Run ID
2024-10-01-01J94RQ8MRVWG9XGHF1RYFE927
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Melanie"> Melanie is a 30-year old college student on a study abroad program. Born in Germany, she is fluent in German and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of German, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Melanie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in German, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Melanie and Hunter as a voice sheet: Melanie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Hunter steps into the cafe and spots the young woman at once, her auburn hair pulled back in a messy bun, an expensive-looking camera resting on the table beside her. He approaches her booth, flashing his badge. "Excuse me miss, I'm Detective Hunter from the Alborn PD. Might I have a word with you?" Melanie looks up from the display on her camera, surprise flashing across her clear green eyes. She opens her mouth to respond, then pauses, realizing he's speaking in English. Sighing softly, she switches to her native German. "I...what? I'm sorry, I don't really understand..." She replies hesitantly, hands fluttering nervously over the camera. Hunter squints at her, not comprehending a word she's saying. He whips out a notebook, sketching a crude drawing of the symbol that's been appearing around town. "This. See this symbol? Keep an eye out for it, will you?"He taps the page for emphasis, then stands back expectantly. Melanie's eyes widen as she stares at the notebook. "Oh Gott," she breathes, voice trembling, "That's the same one...the one on the girl's van the other night..." Her hands shake as she speaks rapidly in German, her worst fears confirmed. Hunter watches her intently, managing to pick up the emotion if not the meaning. "Whoa, slow down," he says, holding up a hand. "Let's start again, okay? Tell me your name, what you saw..." A police scanner crackles to life on his belt, spitting out a string of garbled emergency codes. Hunter holds up a finger to Melanie, absently frowning as he tilts his ear to listen. "Code 3, we have reports of another abduction on Oak St...All units respond..." Cursing under his breath, Hunter jabs a finger at Melanie. "Gotta go. You, wait here." He hurries to the door, pausing to call over his shoulder, "Tell nobody, got it?"
id
"gen-1727810741-8E4B9lOahyxkXnOJixGe"
total_cost
0.0002825
upstream_id
"chatcmpl-ffbb428e-5e37-4c23-81fd-6fd337acbcac"
created_at
"2024-10-01T19:26:02.84711+00:00"
Tuesday, October 1st, 2024 at 7:26:02 PM GMT+00:00
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
43
moderation_latency
(null)
generation_time
21590
finish_reason
"stop"
tokens_prompt
271
tokens_completion
426
native_tokens_prompt
278
native_tokens_completion
426
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0002825
Evaluation details
Result
Evaluator
Details
Meta Data
33.3333%
Parse dialogue
n/a
dialogue
0
name
"Melanie"
dialogue
"That's the same one...the one on the girl's van the other night..."
languages
eng
1
deu
0.7853015382111175
1
name
"Hunter"
dialogue
"Let's start again, okay? Tell me your name, what you saw..."
languages
eng
1
deu
0.970787468247248
2
name
"Hunter"
dialogue
"Tell nobody, got it?"
languages
eng
1
deu
0.8803869832893579
passes
true