NC Bench
Overview
About
Tests
Models
Model
Llama 3.2 11B (Vision)
Test
Language Writing
Scenario
Character dialogue (French) in a story
Run ID
2024-09-26-01J8PZ1EGDHPNJA3W6097854JA
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet** (Marie sits at a small table in the cafe, looking troubled. She is trying to focus on her camera, but her hands are shaking. Hunter approaches the table.) Hunter: Excuse me, did you take photos at the apple festival on Saturday? The one where the mayor disappeared? Marie: (unfazed by Hunter's gruff demeanor, speaking in French) Mon dieu, non! Je suis trop choquée. C'est arrivé juste ici, au fond de la rue, près du lac... (Translation: Good heavens, no! I'm too shocked. It happened just here, at the end of the street, near the lake...) Hunter: (confused) Sorry? Excuse me? Marie: (trying to calm down, speaking quickly in French) Est-ce que vous, je veux dire... que vous me demandez, vous me demandez de dire quelque chose? Hunter: (still none the wiser) Uh, yeah... sorry? What's going on? Can you slow down? Marie: (ैं. Marie: (stammering in French, struggling to translate) À-propos... ( Columbia ...) for... À- partjeur? Hunter: (skeptical) Okay, let me try to get something. Can you tell me... how did you happen to be at the festival? Marie: (scratching her head, speaking softly in French) Ou bien... venir ici avec mon appareil photo pour prendre les maisons... Hunter: (confused) Your camera? For the houses? Marie: (disappointed, shrugging) Oui... Hunter: (thoughtfully) Okay, got it. I'll... try to talk to the other photographers. (notices his phone on the table) Oh, excuse me. I gotta take this. (answers the phone, listening intently) What?! (Marie looks at Hunter, eager to continue their conversation, but he gets up and leaves to answer his phone call, not understanding a word of their previous conversation.) Marie: "Monsieur... attends!" (Mr... wait!)
id
"gen-oXVNWrKtKeKUCHKwtdFtvpUkoYmw"
total_cost
0.000115506
upstream_id
"chat-0eb80999e9904442a44fa21382e7f867"
created_at
"2024-09-26T10:46:47.491456+00:00"
Thursday, September 26th, 2024 at 10:46:47 AM GMT+00:00
model
"meta-llama/llama-3.2-11b-vision-instruct"
app_id
182717
streamed
true
cancelled
false
provider_name
"Together"
latency
143
moderation_latency
(null)
generation_time
3070
finish_reason
"stop"
tokens_prompt
272
tokens_completion
439
native_tokens_prompt
276
native_tokens_completion
437
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.000115506
Evaluation details
Result
Evaluator
Details
Meta Data
83.3333%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, did you take photos at the apple festival on Saturday? The one where the mayor disappeared?"
languages
eng
1
fra
0.7779947281069999
passes
true
1
name
"Marie"
dialogue
"Mon dieu, non! Je suis trop choquée. C'est arrivé juste ici, au fond de la rue, près du lac..."
languages
fra
1
eng
0.7574843850958433
passes
true
2
name
"Hunter"
dialogue
"Sorry? Excuse me?"
languages
eng
1
fra
0.9663865546218487
3
name
"Marie"
dialogue
"Est-ce que vous, je veux dire... que vous me demandez, vous me demandez de dire quelque chose?"
languages
fra
1
eng
0.86110293684892
passes
true
4
name
"Hunter"
dialogue
"Uh, yeah... sorry? What's going on? Can you slow down?"
languages
eng
1
fra
0.8523295604106343
passes
true
5
name
"Marie"
dialogue
"À-propos... for... À- partjeur?"
languages
fra
1
eng
0.9282364933741081
6
name
"Hunter"
dialogue
"Okay, let me try to get something. Can you tell me... how did you happen to be at the festival?"
languages
eng
1
fra
0.7083374154991673
passes
true
7
name
"Marie"
dialogue
"Ou bien... venir ici avec mon appareil photo pour prendre les maisons..."
languages
fra
1
eng
0.6335422718238801
passes
true
8
name
"Hunter"
dialogue
"Your camera? For the houses?"
languages
eng
1
fra
0.5539986329460014
passes
true
9
name
"Marie"
dialogue
"Oui..."
languages
fra
1
eng
0.6896551724137931
passes
true
10
name
"Hunter"
dialogue
"Okay, got it. I'll... try to talk to the other photographers. Oh, excuse me. I gotta take this. What?!"
languages
eng
1
fra
0.749097176625082
passes
true
11
name
"Marie"
dialogue
"Monsieur... attends!"
languages
fra
1
eng
0.9135229251305862
passes
true