NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2BRQX1C511YWY866J4M9D
Run Details
Model:
GPT-4o, Aug. 6th (temp=0)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
6.0s
Cost:
$0.004870
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet:** *Setting: A quaint cafe in Alborn, MN. The aroma of freshly brewed coffee fills the air as Marie sits at a small table, her camera resting in her hands. She appears deep in thought, her eyes occasionally glancing at the photos she has taken. Hunter, with his usual determined stride, approaches her table.* --- **Hunter:** "Excuse me, miss. I'm Detective Hunter. I noticed you were at the scene of the recent event in town. Can I ask you a few questions?" **Marie:** "Je suis désolée, je ne parle pas anglais. C'était terrifiant, je ne sais pas quoi faire." (I'm sorry, I don't speak English. It was terrifying, I don't know what to do.) **Hunter:** *pauses, looking perplexed* "Uh, okay. I don't speak French, but maybe you can help me. Did you see anything unusual?" **Marie:** *gestures with her hands, trying to convey her emotions* "Les lumières... elles étaient étranges. Et puis, il y a eu ce bruit..." (The lights... they were strange. And then, there was this noise...) **Hunter:** *nodding, trying to understand* "Lights? Noise? Can you show me on your camera?" **Marie:** *nods, scrolling through her photos, showing him an image* "Ici, regardez. C'est ce que j'ai vu." (Here, look. This is what I saw.) **Hunter:** *leans in, examining the photo* "Hmm, interesting. This might be helpful. Thank you." *Marie looks relieved, but still shaken. Hunter's phone rings, interrupting their exchange.* **Hunter:** *answers the call, then turns to Marie* "I have to go. Thank you for your help. I'll be in touch." **Marie:** "D'accord, au revoir." (Okay, goodbye.) *Hunter nods and quickly exits the cafe, leaving Marie to ponder the mysterious events captured in her camera.*
id
"gen-9aUVt3NmeeoyPjNssigPTAOLZj85"
total_cost
0.00487
upstream_id
"chatcmpl-9uNXhyZCYsZvPMshyUluNSZBQeI5I"
created_at
"2024-08-09T17:18:18.905678+00:00"
8/9/2024, 5:18:18 PM
model
"openai/gpt-4o-2024-08-06"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
5692
moderation_latency
193
generation_time
0
finish_reason
"stop"
tokens_prompt
272
tokens_completion
436
native_tokens_prompt
268
native_tokens_completion
420
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00487
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. I'm Detective Hunter. I noticed you were at the scene of the recent event in town. Can I ask you a few questions?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8097763430943047
tl
0.6597938144329897
it
0.6029836381135707
fr
0.5993200582807188
nl
0.5523602821486707
ro
0.5436946902654868
ca
0.5346869712351946
no
0.5333710407239819
pt
0.5200698080279231
da
0.5106761565836299
sv
0.5006053268765134
es
0.4649805447470817
sl
0.416135881104034
sq
0.41447835344215755
de
0.3947175348495965
cs
0.39204126750184226
yo
0.3838685586258402
sk
0.374051593323217
pl
0.3574766355140187
hr
0.3559718969555035
ms
0.3032094594594595
az
0.26273458445040215
lt
0.25541516245487367
tr
0.25541516245487367
lv
0.233983286908078
is
0.21577946768060838
hu
0.21278625954198474
et
0.2112810707456979
fi
0.14330218068535827
vi
0.13883089770354906
be
0.0782122905027933
eu
0.0782122905027933
uk
0.07094594594594594
reliable
true
passes
true
1
name
"Marie"
dialogue
"Je suis désolée, je ne parle pas anglais. C'était terrifiant, je ne sais pas quoi faire."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8075513196480938
ca
0.5147874306839186
lt
0.4980879541108987
sl
0.4980879541108987
sq
0.4631901840490798
nl
0.4593202883625128
en
0.45255474452554745
cs
0.41471571906354515
hr
0.3979357798165138
it
0.3794326241134752
ro
0.3794326241134752
sk
0.3526510480887793
lv
0.32778489116517284
pl
0.29245283018867924
es
0.29149797570850206
hu
0.28765264586160105
de
0.2818057455540356
yo
0.2369186046511628
tr
0.22566371681415928
fi
0.21875
ms
0.21875
pt
0.20454545454545453
az
0.20212765957446807
et
0.1935483870967742
da
0.18856259659969088
no
0.18856259659969088
sv
0.18856259659969088
tl
0.12935323383084577
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Uh, okay. I don't speak French, but maybe you can help me. Did you see anything unusual?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7973925299506694
tl
0.5228215767634855
et
0.4295634920634921
sq
0.3991640543364681
es
0.3251173708920188
it
0.32113341204250295
pt
0.3097238895558223
ms
0.2936117936117936
no
0.2909987669543773
ca
0.2721518987341772
cs
0.2414248021108179
fr
0.2374005305039788
tr
0.203601108033241
az
0.1958041958041958
da
0.19467787114845939
hr
0.19467787114845939
sv
0.1935483870967742
pl
0.1797432239657632
ro
0.13533834586466165
sl
0.13273001508295626
yo
0.1314199395770393
nl
0.1274658573596358
fi
0.11538461538461538
sk
0.11128284389489954
be
0.10852713178294573
hu
0.1043613707165109
is
0.0987460815047022
lt
0.0987460815047022
lv
0.0987460815047022
uk
0.0987460815047022
vi
0.0987460815047022
reliable
true
passes
true
3
name
"Marie"
dialogue
"Les lumières... elles étaient étranges. Et puis, il y a eu ce bruit..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8210023866348448
ca
0.5830023828435266
ro
0.5679012345679012
da
0.5278776978417267
no
0.5196706312900274
nl
0.48880233690360275
es
0.48173741362290223
it
0.45764462809917356
en
0.4366952789699571
et
0.4330453563714903
hu
0.39724454649827784
sv
0.3902439024390244
de
0.37796208530805686
pt
0.36286407766990286
fi
0.2857142857142857
pl
0.2857142857142857
cs
0.27884615384615385
sk
0.26675977653631283
hr
0.26264044943820225
az
0.2605633802816902
vi
0.24242424242424243
sl
0.23469387755102045
tr
0.23469387755102045
sq
0.23133235724743775
tl
0.23133235724743775
yo
0.1847826086956522
lt
0.13509060955518945
is
0.125
eu
0.11764705882352942
lv
0.11764705882352942
ms
0.10714285714285714
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Lights? Noise? Can you show me on your camera?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7728993186979561
tl
0.6717724288840263
fr
0.5670995670995671
ca
0.4809688581314879
et
0.453551912568306
it
0.44237918215613387
nl
0.441340782122905
ms
0.43820224719101125
fi
0.42857142857142855
es
0.40119760479041916
pt
0.3548387096774194
ro
0.34497816593886466
cs
0.33774834437086093
sq
0.33035714285714285
no
0.23273657289002556
hr
0.22680412371134018
sl
0.22680412371134018
hi
0.21875
pl
0.21875
sv
0.21465968586387435
az
0.20424403183023873
tr
0.20424403183023873
sk
0.2
yo
0.2
de
0.1935483870967742
reliable
true
passes
true
5
name
"Marie"
dialogue
"Ici, regardez. C'est ce que j'ai vu."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8230714791224345
es
0.503968253968254
ca
0.50199203187251
ro
0.46236559139784944
pt
0.4075829383886255
it
0.37810945273631835
nl
0.2857142857142857
en
0.2492492492492493
tl
0.22839506172839508
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Hmm, interesting. This might be helpful. Thank you."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7980380842469705
tl
0.6365524402907581
no
0.5125348189415042
nl
0.48604992657856094
lt
0.48377581120943947
de
0.4712990936555891
ro
0.4631901840490798
pt
0.46236559139784944
it
0.4615384615384615
hu
0.41666666666666663
es
0.39759036144578314
da
0.3944636678200692
sv
0.39130434782608703
ca
0.38917975567190227
hr
0.3783303730017762
vi
0.37275985663082434
yo
0.3445692883895131
fi
0.3320610687022901
fr
0.31906614785992216
lv
0.317738791423002
ms
0.3110236220472441
cs
0.3069306930693069
eu
0.3027888446215139
is
0.3
pl
0.29292929292929293
sq
0.29292929292929293
sl
0.2842535787321063
sk
0.1764705882352941
tr
0.17061611374407584
et
0.16666666666666666
az
0.1606714628297362
reliable
true
passes
true
7
name
"Hunter"
dialogue
"I have to go. Thank you for your help. I'll be in touch."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8267898383371826
tl
0.6097814776274715
da
0.5941558441558442
yo
0.5709382151029748
pl
0.5471014492753623
no
0.5235069885641677
cs
0.455732946298984
lt
0.42660550458715596
sl
0.4230769230769231
it
0.4113029827315541
hr
0.40476190476190477
sv
0.3951612903225807
ro
0.3822075782537067
vi
0.3567753001715266
sk
0.35233160621761656
lv
0.33035714285714285
hu
0.32915921288014316
nl
0.2937853107344633
ms
0.28977272727272724
az
0.2816091954022989
de
0.2816091954022989
ca
0.2690058479532163
sq
0.2632612966601179
fr
0.2299794661190965
pt
0.18655097613882862
be
0.15730337078651685
tr
0.15730337078651685
et
0.14383561643835618
is
0.14383561643835618
uk
0.14383561643835618
reliable
true
passes
true
8
name
"Marie"
dialogue
"D'accord, au revoir."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8026315789473685
ro
0.6894409937888198
it
0.625
ca
0.620253164556962
en
0.494949494949495
es
0.47735191637630664
nl
0.47735191637630664
tl
0.35344827586206895
cs
0.3023255813953488
da
0.3023255813953488
hr
0.3023255813953488
no
0.3023255813953488
pt
0.3023255813953488
sk
0.3023255813953488
sl
0.3023255813953488
sq
0.3023255813953488
reliable
true
passes
true