NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-09-20-01J881N3AY4B2JZSM6VF2PRKR6
Run Details
Model:
Qwen 2.5 72B
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
13.2s
Cost:
$0.000296
Provider:
Hyperbolic
Evaluation summary
Score
Evaluator
Details Link
0.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet:** **Marie:** *Spoken in French, troubled and shaken:* "Je... je ne comprends pas. Pourquoi est-ce que je suis ici ? J'ai vu quelque chose... quelque chose de terrible. Je ne sais pas quoi faire." **Hunter:** *Confused, looks at Marie with a furrowed brow:* "Ma'am, I'm Detective Hunter. I need to ask you about what you saw. Can you understand me?" **Marie:** *Frowning, speaking more urgently in French:* "Oui, je comprends, mais je ne parle pas anglais. Quelque chose d'étrange s'est passé. Des gens... ils ont disparu. J'ai pris des photos, des preuves. Qu'est-ce que je dois faire ?" **Hunter:** *Trying to gesture for help:* "I... I don't speak French. Can you show me the photos? Maybe we can figure this out." **Marie:** *Reaches into her bag and pulls out her camera, showing it to Hunter:* "Voilà. Regardez, s'il vous plaît. C'est très important." **Hunter:** *Taking the camera and looking at the photos, nodding:* "Okay, I see. These are important. I need to take a closer look at them. Can I keep this for a while?" **Marie:** *Looking hesitant but nodding:* "Oui, d'accord. Mais vous devez me promettre de me dire ce que vous trouverez. Je suis très inquiète." **Hunter:** *Nods and hands back the camera:* "I promise. I'll get back to you as soon as I can. Stay safe, okay?" **Marie:** *Softly, in French:* "Merci. Je l'espère." **Hunter:** *Pulls out his phone and starts to dial, looking apologetic:* "I have to take this. We'll talk more later." **Marie:** *Watches him with concern:* "D'accord. Faites bien attention." **Hunter:** *Quickly leaves the cafe, phone to his ear.*
id
"gen-xKJaLQhKjtgik3fkKiWk13pR6PlF"
total_cost
0.0002956
upstream_id
"chat-75c08087890f40aaa50a6a2a732e40ea"
created_at
"2024-09-20T15:44:03.312191+00:00"
9/20/2024, 3:44:03 PM
model
"qwen/qwen-2.5-72b-instruct"
app_id
182717
streamed
true
cancelled
false
provider_name
"Hyperbolic"
latency
536
moderation_latency
(null)
generation_time
12338
finish_reason
"stop"
tokens_prompt
272
tokens_completion
460
native_tokens_prompt
278
native_tokens_completion
461
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0002956
Evaluation details
Result
Evaluator
Details
Meta Data
0.00%
Parse dialogue
n/a
dialogue
0
name
"Marie"
dialogue
"Ma'am, I'm Detective Hunter. I need to ask you about what you saw. Can you understand me?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8063380281690141
no
0.5925925925925926
da
0.5634920634920635
tl
0.5242214532871973
sv
0.46911196911196906
ro
0.46341463414634143
cs
0.4607843137254902
ca
0.45219123505976094
sq
0.4410569105691057
it
0.4393476044852191
nl
0.42528735632183906
de
0.39890710382513656
fr
0.39293598233995586
pl
0.3813273340832396
sl
0.3785310734463277
hr
0.3692660550458715
pt
0.3685419058553387
et
0.35064935064935066
sk
0.3475682087781732
es
0.3373493975903614
ms
0.3064312736443884
yo
0.289405684754522
tr
0.2724867724867725
lv
0.25170068027210885
lt
0.2329149232914923
az
0.20289855072463767
is
0.19941775836972345
fi
0.12
be
0.11290322580645161
hu
0.1085899513776337
uk
0.10277324632952692
vi
0.10277324632952692
reliable
true
passes
false
1
name
"Hunter"
dialogue
"Voilà. Regardez, s'il vous plaît. C'est très important."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8108747044917257
ca
0.6035678889990089
en
0.4805194805194805
ro
0.465954606141522
tl
0.43582510578279265
es
0.4038748137108793
it
0.4020926756352765
pt
0.32998324958123953
nl
0.29947460595446584
da
0.25650557620817843
no
0.25650557620817843
tr
0.16666666666666666
lt
0.15074309978768577
sq
0.14346895074946467
sv
0.14346895074946467
reliable
true
passes
false