NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2EE97DCAYSFBEWZXFVR4J
Run Details
Model:
GPT-4o, Aug. 6th (temp=1)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
6.8s
Cost:
$0.005670
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
[INT. CAFE - DAY] Marie sits at a small table by the window, the afternoon light casting gentle shadows. She fiddles with her camera, reviewing images of the local apple orchards. Suddenly, Hunter approaches her table with a determined gait. Hunter: "Excuse me, are you the one who's been taking those pictures around town?" Marie looks up, surprised. She quickly realizes he’s addressing her, and her heart races slightly. She nods, clutching her camera, speaking in rapid French. Marie: "Oui, j'ai pris quelques photos... mais je ne sais pas... Le festival, c'était si étrange. Il y avait quelque chose de... bizarre." Hunter: (puzzled) "Uh, I don't understand. Do you speak English?" Marie's brow furrows slightly as she shakes her head, hoping her gestures convey more than her words. Marie: "Non. Désolé, pas anglais." Hunter: (pausing, trying to communicate) "Festival... strange? You saw something? Picture?" Marie hesitates, glancing at her camera. She points at it, then looks toward Hunter, attempting to share her unease through her eyes. Marie: "Oui, photo... étrange. Je ne sais pas... inquiétant." Hunter scratches his head, clearly struggling to understand but sensing the worry in her tone. Hunter: "Okay, uh, 'photo' and 'strange'? Maybe... I can see?" Marie eagerly turns the camera's screen toward him, showing a picture of an unusually dark shadow amidst the vibrant festival scenes. Her voice lowers as she explains with urgent concern. Marie: "C'est différent... quelque chose ne va pas... un homme, il était là, puis parti." Hunter stares at the image, his detective instincts piqued despite the language barrier. He looks back at Marie, trying to decipher as much as he can. Hunter: "Alright, thanks. I'll try to figure this out. Someone could translate later, maybe?" Just then, Hunter's phone rings loudly. He glances at it, then back at Marie with an apologetic nod. Hunter: "I have to go. Important call. Merci, is that right?" Marie nods, catching the one word she understands. Marie: "Oui, merci." Hunter gives a brief smile, one full of understanding and appreciation before briskly leaving the cafe, leaving Marie to ponder the shadowy mystery lurking in Alborn.
id
"gen-76sam9aZXlSI9b4M6M3iAQxkFeuw"
total_cost
0.00567
upstream_id
"chatcmpl-9uNZ7STFzE6UkP7Z6S8V3ALWK7Nvl"
created_at
"2024-08-09T17:19:47.303436+00:00"
8/9/2024, 5:19:47 PM
model
"openai/gpt-4o-2024-08-06"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
6174
moderation_latency
174
generation_time
0
finish_reason
"stop"
tokens_prompt
272
tokens_completion
527
native_tokens_prompt
268
native_tokens_completion
500
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00567
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, are you the one who's been taking those pictures around town?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8015085351329893
tl
0.5928338762214984
fr
0.5210727969348659
sq
0.44258639910813824
no
0.4075829383886255
nl
0.40617577197149646
ro
0.375
et
0.34554973821989526
sv
0.3065187239944522
pl
0.2784992784992785
ca
0.2537313432835821
ms
0.24471299093655588
pt
0.24127465857359634
sl
0.24127465857359634
fi
0.2366412213740458
de
0.2331288343558282
da
0.23076923076923078
yo
0.17627677100494232
es
0.16943521594684383
hr
0.1482112436115843
tr
0.14675767918088736
lv
0.1423670668953688
it
0.13793103448275862
cs
0.13494809688581316
is
0.1243432574430823
sk
0.1243432574430823
reliable
true
passes
true
1
name
"Marie"
dialogue
"Oui, j'ai pris quelques photos... mais je ne sais pas... Le festival, c'était si étrange. Il y avait quelque chose de... bizarre."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8158179848320692
ca
0.5352651722252597
it
0.5200451722190852
es
0.5148401826484018
pt
0.5114942528735632
en
0.5092378752886837
sq
0.47270471464019853
sl
0.4572158365261813
nl
0.45686900958466453
ro
0.4554772581678411
no
0.4026704146170063
tl
0.3902439024390244
cs
0.3862815884476534
da
0.38181818181818183
yo
0.371301775147929
hr
0.36329588014981273
lt
0.3609022556390977
de
0.3511450381679389
sk
0.3122977346278317
sv
0.3122977346278317
hu
0.288107202680067
ms
0.2772108843537415
az
0.2582897033158813
tr
0.24511545293072823
pl
0.23354373309287646
lv
0.21803127874885006
fi
0.15084915084915085
et
0.12551440329218108
vi
0.10337552742616034
hi
0.08306364617044228
eu
0.07608695652173912
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Uh, I don't understand. Do you speak English?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7987616099071208
no
0.6171967020023557
da
0.6117084826762246
sv
0.564343163538874
cs
0.5517241379310345
pl
0.5163690476190477
hr
0.5045731707317073
tl
0.4984567901234568
sk
0.4741100323624596
de
0.4491525423728813
sq
0.4357638888888889
ro
0.4206773618538324
nl
0.3538767395626242
vi
0.3486973947895792
yo
0.336734693877551
pt
0.32432432432432434
ca
0.29805615550755943
is
0.29653679653679654
sl
0.2696629213483146
it
0.26136363636363635
ms
0.181360201511335
be
0.17721518987341772
tr
0.17721518987341772
hu
0.1709183673469388
az
0.16237113402061853
et
0.16237113402061853
lt
0.16237113402061853
lv
0.16237113402061853
uk
0.16237113402061853
reliable
true
passes
true
3
name
"Marie"
dialogue
"Non. Désolé, pas anglais."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7612551159618008
lt
0.5872641509433962
sq
0.5454545454545454
en
0.48377581120943947
ca
0.4807121661721069
it
0.4648318042813456
lv
0.34944237918215615
nl
0.34210526315789475
eu
0.3320610687022901
ms
0.32432432432432434
cs
0.3137254901960784
sl
0.3137254901960784
ro
0.2886178861788618
sk
0.2886178861788618
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Festival... strange? You saw something? Picture?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7915218902015289
tl
0.6815286624203821
fr
0.5951417004048583
ms
0.5833333333333334
da
0.5607613469985359
no
0.5568685376661743
hr
0.5562130177514794
it
0.5502248875562219
sl
0.5419847328244275
nl
0.5391705069124424
cs
0.5370370370370371
de
0.5098039215686274
pt
0.48186528497409326
ro
0.4746059544658494
es
0.46524064171123
sq
0.46428571428571425
ca
0.40711462450592883
sk
0.40711462450592883
lt
0.39148073022312374
sv
0.3877551020408163
et
0.37629937629937626
az
0.3710691823899371
pl
0.3197278911564626
lv
0.3071593533487298
yo
0.19137466307277629
hu
0.18256130790190736
is
0.18256130790190736
fi
0.17355371900826447
tr
0.16434540389972147
reliable
true
passes
true
5
name
"Marie"
dialogue
"Oui, photo... étrange. Je ne sais pas... inquiétant."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7955607476635514
nl
0.5185694635488307
en
0.5165745856353591
ca
0.4992846924177396
cs
0.47526236881559225
sl
0.46236559139784944
hr
0.4479495268138801
lt
0.43820224719101125
sq
0.40476190476190477
tl
0.38811188811188807
da
0.360146252285192
no
0.35064935064935066
ro
0.34210526315789475
de
0.3320610687022901
it
0.3217054263565891
yo
0.317738791423002
pt
0.3069306930693069
sk
0.29292929292929293
lv
0.2273730684326711
tr
0.21171171171171171
pl
0.20634920634920634
fi
0.2009132420091324
ms
0.1935483870967742
hu
0.1898148148148148
hi
0.18032786885245902
sv
0.18032786885245902
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Okay, uh, 'photo' and 'strange'? Maybe... I can see?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7422680412371134
tl
0.5289855072463767
ca
0.473257698541329
da
0.4491525423728813
no
0.4472789115646259
fr
0.4425385934819897
et
0.417562724014337
sv
0.3981481481481482
cs
0.3725868725868726
yo
0.36399217221135033
tr
0.3114406779661017
az
0.3010752688172043
hr
0.2995689655172413
sq
0.2919389978213508
pl
0.2793791574279379
it
0.26136363636363635
de
0.2243436754176611
ro
0.21686746987951808
nl
0.214975845410628
pt
0.19154228855721395
sk
0.181360201511335
be
0.17721518987341772
hu
0.1709183673469388
is
0.16237113402061853
lt
0.16237113402061853
lv
0.16237113402061853
uk
0.16237113402061853
vi
0.16237113402061853
reliable
true
passes
true
7
name
"Marie"
dialogue
"C'est différent... quelque chose ne va pas... un homme, il était là, puis parti."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8324703015534571
it
0.6255956432947583
ca
0.5733126454615981
ro
0.5682888540031397
en
0.502262443438914
sq
0.40796555435952636
es
0.3888888888888889
lt
0.3861607142857143
lv
0.3834080717488789
no
0.3663594470046083
tr
0.36342592592592593
et
0.36046511627906974
da
0.3226600985221675
yo
0.3150684931506849
nl
0.3064312736443884
ms
0.30467762326169406
az
0.26174496644295303
sl
0.26174496644295303
hr
0.24657534246575344
fi
0.23184357541899442
de
0.2231638418079096
vi
0.22206506364922204
cs
0.20863309352517986
hu
0.20863309352517986
sv
0.207492795389049
pt
0.1499227202472952
eu
0.12698412698412698
tl
0.12698412698412698
pl
0.11290322580645161
sk
0.10277324632952692
reliable
true
passes
true
8
name
"Hunter"
dialogue
"Alright, thanks. I'll try to figure this out. Someone could translate later, maybe?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8062015503875969
tl
0.5209059233449477
fr
0.46808510638297873
nl
0.46808510638297873
no
0.466537342386033
da
0.4234800838574424
de
0.39956331877729256
ro
0.36342592592592593
pt
0.3514150943396226
pl
0.3381468110709988
hr
0.3159203980099502
vi
0.29757343550446996
es
0.289405684754522
yo
0.27726675427069647
it
0.27344782034346105
sk
0.26174496644295303
cs
0.25675675675675674
lv
0.2557510148849797
sl
0.24657534246575344
lt
0.23717059639389734
ca
0.21652421652421652
sq
0.21316165951359084
sv
0.20863309352517986
hi
0.12140575079872205
eu
0.1085899513776337
tr
0.10277324632952692
az
0.09688013136288999
et
0.09688013136288999
ms
0.09688013136288999
reliable
true
passes
true
9
name
"Hunter"
dialogue
"I have to go. Important call. Merci, is that right?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.821337417049515
tl
0.707112970711297
da
0.6323529411764706
fr
0.6058558558558559
ca
0.5967741935483871
yo
0.5857988165680473
it
0.5783132530120482
no
0.571078431372549
pl
0.56575682382134
ro
0.5104895104895105
cs
0.4720965309200603
es
0.4479495268138801
pt
0.4479495268138801
hr
0.42148760330578516
lt
0.40476190476190477
nl
0.396551724137931
hu
0.38704028021015763
tr
0.3848857644991213
sv
0.36936936936936937
sk
0.36823104693140796
sq
0.36479128856624315
sl
0.35185185185185186
lv
0.34579439252336447
is
0.3069306930693069
de
0.29006085192697767
be
0.16666666666666666
ms
0.16467780429594273
az
0.15254237288135594
et
0.15254237288135594
uk
0.15254237288135594
vi
0.15254237288135594
reliable
true
passes
true
10
name
"Marie"
dialogue
"Oui, merci."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7819767441860466
it
0.6938775510204083
es
0.5370370370370371
pl
0.5370370370370371
pt
0.5370370370370371
ca
0.5222929936305732
cs
0.506578947368421
de
0.4791666666666667
en
0.4791666666666667
ms
0.4791666666666667
sl
0.4791666666666667
tl
0.4791666666666667
tr
0.4791666666666667
reliable
true
passes
true