NC Bench
Overview
About
Tests
Models
Model
Gemma 2 9B
Test
Language Writing
Scenario
Character dialogue (Spanish) in a story
Run ID
2024-08-09-01J4W18NDXA87016Z5QZYAA8WE
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Maria"> Maria is a 30-year old college student on a study abroad program. Born in Spain, she is fluent in Spanish and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Spanish, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Maria is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Spanish, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Maria and Hunter as a voice sheet: Maria: "..." Hunter: "..."
Model response
Messages
Meta
AI
## Scene: Cafe in Alborn **(Maria sits at a table outside a quaint cafe, sunlight dappling on her face. She's absorbed, reviewing photos on her camera. Hunter approaches cautiously, his face serious.)** **Hunter:** Excuse me, ma'am. **(Maria looks up, startled. She smiles politely but her eyes are troubled.)** **Maria:** Hola. **Hunter:** (Confused) Uh... Hello? **(Maria gestures to her camera.)** **Maria:** (Spanish) Estoy... estoy revisando las fotos. **Hunter:** (Shaking his head) I'm sorry, I don't... **(He gestures to himself) ** **Hunter:** Hunter. Detective. **(Maria takes a deep breath, her eyes flitting around nervously.)** **Maria:** (Spanish) Detective... ¿Sobre qué usted quiere hablar? **Hunter:** (Confused) About the... festival. Did you see anything unusual? **(Maria's eyes widen, she looks back at her camera, then back at Hunter with a mix of fear and confusion.)** **Maria:** (Spanish) ¡Sí! Muy extraño. Hubo... Hubo personas... desaparecidas. **(She trails off, her voice catching.)** **Hunter:** (Leaning in) Disappeared? You saw something? **(Maria nods, unable to speak. She points to the camera, then looks pleadingly at Hunter.)** **Maria:** (Spanish) Las fotos... las fotos pueden ayudar. **(Hunter frowns, he doesn't understand a word. He pulls out his phone and taps it, then looks at Maria.)** **Hunter:** (Frustrated) I'll... I'll get someone who speaks Spanish. **(He sighs, then checks his watch.)** **Hunter:** Wait here. **(Hunter walks away. Maria watches him go, her face etched with fear.)**
id
"gen-5y8j4so02PlZiOVud0fSFYp5fRAY"
total_cost
0.000144
upstream_id
"15e06d36-ed9e-4bf9-ac85-37c2271dc813"
created_at
"2024-08-09T16:59:07.517689+00:00"
Friday, August 9th, 2024 at 4:59:07 PM GMT+00:00
model
"google/gemma-2-9b-it"
app_id
182717
streamed
false
cancelled
false
provider_name
"Fireworks"
latency
4294
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
288
tokens_completion
415
native_tokens_prompt
294
native_tokens_completion
426
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.000144
Evaluation details
Result
Evaluator
Details
Meta Data
69.2308%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, ma'am."
languages
spa
1
eng
0.7884344146685472
1
name
"Maria"
dialogue
"Hola."
languages
spa
1
eng
0.5092436974789916
passes
true
2
name
"Hunter"
dialogue
"Uh... Hello?"
languages
eng
1
spa
0.9461952344350499
3
name
"Maria"
dialogue
"Estoy... estoy revisando las fotos."
languages
spa
1
eng
0.8118668596237337
passes
true
4
name
"Hunter"
dialogue
"I'm sorry, I don't..."
languages
eng
1
spa
0.7151898734177216
passes
true
5
name
"Hunter"
dialogue
"Hunter. Detective."
languages
eng
1
spa
0.8316690442225392
passes
true
6
name
"Maria"
dialogue
"Detective... ¿Sobre qué usted quiere hablar?"
languages
eng
1
spa
0.9137273593898951
7
name
"Hunter"
dialogue
"About the... festival. Did you see anything unusual?"
languages
eng
1
spa
0.6806893995552261
passes
true
8
name
"Maria"
dialogue
"¡Sí! Muy extraño. Hubo... Hubo personas... desaparecidas."
languages
spa
1
eng
0.7704988686563947
passes
true
9
name
"Hunter"
dialogue
"Disappeared? You saw something?"
languages
eng
1
spa
0.49441860465116283
passes
true
10
name
"Maria"
dialogue
"Las fotos... las fotos pueden ayudar."
languages
spa
1
eng
0.8431309389595748
passes
true
11
name
"Hunter"
dialogue
"I'll... I'll get someone who speaks Spanish."
languages
eng
1
spa
0.9433448787728848
12
name
"Hunter"
dialogue
"Wait here."
languages
eng
1
spa
0.7505376344086021
passes
true