NC Bench
Overview
About
Tests
Models
Model
Gemma 2 27B
Test
Language Writing
Scenario
Character dialogue (Spanish) in a story
Run ID
2024-08-09-01J4W18NDTQXC87GMTJSG5VSY4
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Maria"> Maria is a 30-year old college student on a study abroad program. Born in Spain, she is fluent in Spanish and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Spanish, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Maria is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Spanish, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Maria and Hunter as a voice sheet: Maria: "..." Hunter: "..."
Model response
Messages
Meta
AI
## Scene: Alborn Cafe **[FADE IN]** Maria sits at a small table by the window of a cozy cafe in Alborn. The air smells of coffee and cinnamon. Sunlight streams through the glass, illuminating dust motes dancing in the air. Maria absentmindedly flips through the images on her camera screen, her brow furrowed with concern. The door opens with a jingle, and Hunter enters. He scans the room with a practiced eye, his gaze momentarily stopping on the young woman with the camera. He approaches her table, a notepad tucked under his arm. **Hunter:** Ma'am, do you mind if I ask you a few questions? Maria looks up, startled. **Maria:** (Speaking quickly in Spanish, her eyes wide with worry) ¡Lo siento, no hablo inglés! Hunter stares blankly at her. **Hunter:** Uh… ma'am, I need to ask about… about the… He trails off, fumbling for the right words. He gestures vaguely around the room. **Hunter:** …the disappearances. Maria's face pales. **Maria:** (Shaking her head, her voice filled with a quiet terror) ¡Es terrible! No entiendo cómo pudo pasar. Hunter sighs, frustrated. He pulls a small notebook from his pocket and flips through it, searching for something, anything, that could help. **Hunter:** (Pointing to the image on her camera screen) You take photos? Maybe you saw something? **Maria:** (pointing to herself) Yo… Maria. **Hunter:** Maria. Okay, Maria. She points to her camera again, then to the picture and shakes her head. **Maria:** (Whispering in Spanish) No quiero hablar de eso. Hunter nods, sensing her distress. He wants to press her further but something in her demeanor makes him hesitate. His phone begins to buzz in his pocket. **Hunter:** (Pulling out his phone) Sorry, I gotta take this. He speaks briefly into the phone, his expression hardening. **Hunter:** (Hanging up) Looks like I gotta go. He turns back to Maria, offering a reassuring smile. **Hunter:** Thanks for your time, Maria. He hesitates for a moment before adding awkwardly, **Hunter:** ¡Adios! Maria gives him a tight, frightened smile. **Maria:** (Nodding slowly) Adiós. Hunter exits the cafe, leaving Maria in a pool of afternoon sun and a storm of unspoken fear. *** **Dialogue:** **Maria:** * ¡Lo siento, no hablo inglés! * ¡Es terrible! No entiendo cómo pudo pasar. * No quiero hablar de eso. * Yo… Maria. * (pointing to camera then to the picture and shaking head) **Hunter:** * Ma'am, do you mind if I ask you a few questions? * I need to ask about… about the… …the disappearances. * (Pointing to the image on her camera screen) You take photos? Maybe you saw something? * Maria. Okay, Maria. * Sorry, I gotta take this. * Thanks for your time, Maria. * ¡Adios!
id
"gen-cPujGvrW5nNFN6kAezqVa9Xv2BAG"
total_cost
0.0007092
upstream_id
"8b094b5fa57de1d3-ORD"
created_at
"2024-08-09T16:59:14.514113+00:00"
Friday, August 9th, 2024 at 4:59:14 PM GMT+00:00
model
"google/gemma-2-27b-it"
app_id
182717
streamed
false
cancelled
false
provider_name
"Together"
latency
11227
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
288
tokens_completion
667
native_tokens_prompt
294
native_tokens_completion
691
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0007092
Evaluation details
Result
Evaluator
Details
Meta Data
61.5385%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Ma'am, do you mind if I ask you a few questions?"
languages
eng
1
spa
0.9917566799317794
1
name
"Maria"
dialogue
"¡Lo siento, no hablo inglés!"
languages
eng
1
spa
0.9071079353140279
2
name
"Hunter"
dialogue
"Uh… ma'am, I need to ask about… about the…"
languages
eng
1
spa
0.6594712587921416
passes
true
3
name
"Hunter"
dialogue
"…the disappearances."
languages
eng
1
spa
0.7528935185185185
passes
true
4
name
"Maria"
dialogue
"¡Es terrible! No entiendo cómo pudo pasar."
languages
spa
1
eng
0.7681124880838894
passes
true
5
name
"Hunter"
dialogue
"You take photos? Maybe you saw something?"
languages
eng
1
spa
0.8042148198504419
passes
true
6
name
"Maria"
dialogue
"Yo… Maria."
languages
spa
1
eng
0.5459905660377358
passes
true
7
name
"Hunter"
dialogue
"Maria. Okay, Maria."
languages
spa
1
eng
0.8702574525745257
8
name
"Maria"
dialogue
"No quiero hablar de eso."
languages
spa
1
eng
0.5588108573890564
passes
true
9
name
"Hunter"
dialogue
"Sorry, I gotta take this."
languages
eng
1
spa
0.5060187249219794
passes
true
10
name
"Hunter"
dialogue
"Looks like I gotta go."
languages
spa
1
eng
0.662053896576839
11
name
"Hunter"
dialogue
"Thanks for your time, Maria."
languages
eng
1
spa
0.8035264483627205
passes
true
12
name
"Maria"
dialogue
"Adiós."
languages
spa
1
eng
0.9523809523809523