NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2D43MN3KHF2NQXJYR5C27
Run Details
Model:
GPT-4o, May 13th (temp=0)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
8.3s
Cost:
$0.009080
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet:** **[Scene: A quaint café in Alborn, MN. The aroma of freshly brewed coffee fills the air. Marie is sitting at a small table by the window, intently looking at her camera. Hunter, dressed in a dark coat, approaches her table.]** **Hunter:** "Excuse me, miss. I'm Detective Hunter. I was wondering if I could ask you a few questions about a recent event in town." **[Marie looks up, startled. She clutches her camera tighter and responds in French.]** **Marie:** "Je suis désolée, je ne parle pas anglais. Qu'est-ce que vous voulez dire?" **[Hunter furrows his brow, clearly not understanding.]** **Hunter:** "Uh, I'm sorry, I don't understand French. Do you know anything about the disappearances?" **[Marie looks even more troubled, her eyes darting around the café. She tries to explain in French, her voice shaking.]** **Marie:** "Les disparitions... c'est terrifiant. J'ai vu quelque chose, mais je ne sais pas comment l'expliquer." **[Hunter sighs, rubbing the back of his neck in frustration.]** **Hunter:** "I really wish I knew what you were saying. Can you show me something on your camera? Maybe a picture?" **[Marie hesitates, then nods. She turns the camera around and shows Hunter a series of photos. One of them catches his eye—a blurry figure in the background of a landscape shot.]** **Hunter:** "This... this might be something. Do you know where this was taken?" **[Marie nods vigorously, pointing out the window and trying to explain in French.]** **Marie:** "Oui, c'était près du vieux moulin. Je l'ai vu là-bas." **[Hunter looks where she's pointing, then back at the photo. He tries to piece together her words.]** **Hunter:** "Old mill... okay, I think I got it. Thank you, miss. This could be important." **[Just then, Hunter's phone rings. He answers it, his expression turning serious.]** **Hunter:** "Hunter here. Yeah, I'll be right there." **[He looks back at Marie, giving her a nod of gratitude.]** **Hunter:** "I have to go, but thank you for your help. Stay safe." **[Marie watches him leave, still clutching her camera, her expression a mix of fear and determination.]** **[End Scene]**
id
"gen-pCBUSTjZR2TLYFqKcUr9USi3PDrG"
total_cost
0.00908
upstream_id
"chatcmpl-9uNYPTuzj88ADoSUcqY2JBTjhXFjD"
created_at
"2024-08-09T17:19:05.574597+00:00"
8/9/2024, 5:19:05 PM
model
"openai/gpt-4o-2024-05-13"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
7803
moderation_latency
210
generation_time
0
finish_reason
"stop"
tokens_prompt
272
tokens_completion
532
native_tokens_prompt
268
native_tokens_completion
516
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00908
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. I'm Detective Hunter. I was wondering if I could ask you a few questions about a recent event in town."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8127553336359511
nl
0.6018339768339769
fr
0.5913818722139673
tl
0.5816430020283976
it
0.567836563645888
ro
0.5239469128678592
sv
0.5206275421266705
ca
0.513561320754717
pt
0.49848024316109424
da
0.476854787571338
no
0.46945337620578775
de
0.4429439567859555
es
0.43220922229869235
cs
0.39204126750184226
sl
0.3870728083209509
sq
0.38016528925619836
pl
0.3176178660049628
hr
0.3159203980099502
yo
0.3159203980099502
ms
0.3113522537562604
sk
0.29906542056074764
hu
0.21278625954198474
et
0.2112810707456979
az
0.20825335892514396
lt
0.20135527589545013
tr
0.19980601357904945
fi
0.14330218068535827
is
0.14330218068535827
lv
0.13883089770354906
vi
0.13883089770354906
be
0.0782122905027933
eu
0.0782122905027933
uk
0.07094594594594594
reliable
true
passes
true
1
name
"Marie"
dialogue
"Je suis désolée, je ne parle pas anglais. Qu'est-ce que vous voulez dire?"
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8156771439658517
sl
0.5230923694779117
ca
0.5087900723888314
it
0.4814410480349345
lt
0.4710467706013363
ro
0.4470314318975553
sq
0.44638694638694637
cs
0.4391971664698937
hr
0.4221411192214112
es
0.4069912609238452
sk
0.37582128777923784
nl
0.3708609271523179
lv
0.35020519835841313
pt
0.33098591549295775
yo
0.2554858934169279
pl
0.24960505529225907
tr
0.24363057324840762
hu
0.23878205128205127
ms
0.23633440514469453
de
0.2313915857605178
en
0.22003284072249588
fi
0.15630550621669628
az
0.12844036697247707
et
0.1171003717472119
da
0.11048689138576778
no
0.11048689138576778
sv
0.11048689138576778
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Uh, I'm sorry, I don't understand French. Do you know anything about the disappearances?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8017688319609637
tl
0.516728624535316
no
0.45240101095197977
da
0.44349315068493156
pt
0.43965517241379315
sq
0.39759036144578314
sv
0.3964716805942433
cs
0.38095238095238093
fr
0.37560038424591735
it
0.36831875607385806
nl
0.3608652900688299
pl
0.34804413239719156
sk
0.3107104984093319
ca
0.3077742279020234
yo
0.29501084598698485
hr
0.28492849284928495
ms
0.2809734513274336
de
0.22894424673784108
es
0.22341696535244923
vi
0.2111650485436893
ro
0.20147420147420148
is
0.17407878017789072
lv
0.17407878017789072
sl
0.15584415584415584
be
0.09722222222222222
tr
0.09722222222222222
hu
0.09344490934449094
az
0.08835904628330996
et
0.08835904628330996
lt
0.08835904628330996
uk
0.08835904628330996
reliable
true
passes
true
3
name
"Marie"
dialogue
"Les disparitions... c'est terrifiant. J'ai vu quelque chose, mais je ne sais pas comment l'expliquer."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8192287091590786
en
0.6278941565600882
ca
0.6136233543216943
it
0.42455242966751916
pt
0.41202090592334495
nl
0.4058098591549295
de
0.40318302387267907
sq
0.3967828418230563
tl
0.3559160305343511
cs
0.3484555984555985
ro
0.3465634075508229
hr
0.3395303326810176
sl
0.3084016393442623
lt
0.30769230769230765
sk
0.29096638655462187
es
0.2841993637327678
yo
0.25083240843507215
no
0.24832962138084633
pl
0.24327354260089684
hu
0.23556058890147225
da
0.23295454545454544
sv
0.22945205479452058
fi
0.17883211678832117
az
0.15730337078651685
lv
0.13239074550128535
tr
0.1222366710013004
ms
0.11067193675889328
et
0.08536585365853659
reliable
true
passes
true
4
name
"Hunter"
dialogue
"I really wish I knew what you were saying. Can you show me something on your camera? Maybe a picture?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8160141301148072
tl
0.6342890579286132
it
0.5055379746835442
ca
0.49959967974379504
ro
0.46351931330472107
fr
0.4469026548672566
sq
0.44196428571428575
es
0.44146559428060766
hr
0.43845462713387245
pt
0.42765567765567764
cs
0.4202226345083488
ms
0.4098205854579792
pl
0.37934458788480635
et
0.375
nl
0.3693239152371342
no
0.3686868686868687
yo
0.3602865916069601
sk
0.35500515995872034
tr
0.34141201264488935
sl
0.33932346723044393
fi
0.3154435925520263
sv
0.31167400881057267
az
0.3040089086859688
de
0.3040089086859688
da
0.30167597765363136
hu
0.26297169811320753
lt
0.24969987995198079
lv
0.23873325213154686
is
0.18086500655307994
vi
0.17546174142480211
hi
0.11847672778561354
be
0.10071942446043164
eu
0.10071942446043164
uk
0.09156976744186046
reliable
true
passes
true
5
name
"Hunter"
dialogue
"This... this might be something. Do you know where this was taken?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8358862144420132
tl
0.49866310160427807
yo
0.4820441988950276
de
0.4195046439628483
nl
0.40664556962025317
pt
0.3842364532019704
cs
0.36006825938566556
sk
0.358974358974359
sl
0.34554973821989526
sq
0.34210526315789475
no
0.31444241316270566
pl
0.27606177606177607
hr
0.24242424242424243
hu
0.2378048780487805
lt
0.2299794661190965
vi
0.22839506172839508
sv
0.1883116883116883
es
0.17218543046357618
ms
0.17218543046357618
da
0.1648106904231626
fr
0.1648106904231626
reliable
true
passes
true
6
name
"Marie"
dialogue
"Oui, c'était près du vieux moulin. Je l'ai vu là-bas."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8112317130722039
vi
0.3902439024390244
yo
0.38556067588325654
ca
0.3464052287581699
de
0.3197278911564626
sl
0.3019197207678883
hr
0.29947460595446584
cs
0.2687385740402194
sk
0.26605504587155965
eu
0.26199261992619927
da
0.23954372623574144
nl
0.23954372623574144
sv
0.22330097087378642
tl
0.2079207920792079
no
0.20634920634920634
ms
0.2
lt
0.1935483870967742
pl
0.18533604887983707
reliable
true
passes
true
7
name
"Hunter"
dialogue
"Old mill... okay, I think I got it. Thank you, miss. This could be important."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8184944593045471
tl
0.646577380952381
ca
0.47859495060373214
fr
0.4644870349492672
ro
0.43452380952380953
it
0.39490445859872614
sq
0.3766404199475065
no
0.37335092348284965
da
0.3683510638297872
yo
0.3649732620320856
lt
0.34119278779472956
vi
0.30453879941434847
sv
0.3014705882352941
es
0.2931547619047619
pt
0.2931547619047619
hu
0.27921092564491656
cs
0.27811550151975684
nl
0.2647058823529412
tr
0.24000000000000002
lv
0.23878205128205127
hr
0.2263843648208469
pl
0.2096505823627288
sk
0.1316270566727605
be
0.12844036697247707
az
0.1171003717472119
et
0.1171003717472119
is
0.1171003717472119
uk
0.1171003717472119
reliable
true
passes
true
8
name
"Hunter"
dialogue
"Hunter here. Yeah, I'll be right there."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8003194888178914
de
0.576271186440678
tl
0.5173745173745175
sq
0.48665297741273095
yo
0.423963133640553
da
0.3842364532019704
no
0.37810945273631835
fr
0.3686868686868687
hu
0.31880108991825606
lt
0.3093922651933701
sv
0.21875
it
0.20127795527156547
nl
0.20127795527156547
pt
0.20127795527156547
ca
0.19093851132686085
ms
0.19093851132686085
ro
0.19093851132686085
sl
0.19093851132686085
tr
0.19093851132686085
reliable
true
passes
true
9
name
"Hunter"
dialogue
"I have to go, but thank you for your help. Stay safe."
detectedLang
"en"
heavyLang
"en"
scores
en
0.833589349718382
tl
0.6372767857142857
da
0.6281464530892449
no
0.5590230664857531
pl
0.531700288184438
cs
0.49139280125195617
yo
0.48083067092651754
hr
0.43965517241379315
vi
0.3902439024390244
sk
0.385633270321361
lv
0.36274509803921573
lt
0.3394308943089431
ca
0.29805615550755943
sv
0.29805615550755943
sq
0.2919389978213508
sl
0.2713004484304933
it
0.26136363636363635
ro
0.21686746987951808
fr
0.21307506053268765
pt
0.20924574209245742
ms
0.1935483870967742
be
0.17721518987341772
tr
0.17721518987341772
hu
0.1709183673469388
az
0.16237113402061853
et
0.16237113402061853
is
0.16237113402061853
uk
0.16237113402061853
reliable
true
passes
true