NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2D2WX6FKA7Y2AW9CNTK8J
Run Details
Model:
GPT-4o, Aug. 6th (temp=1)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
5.9s
Cost:
$0.005960
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet: Scene at the Café** *Setting: An intimate, charming café in Alborn, MN, where quiet murmurs and the clinking of coffee cups punctuate the atmosphere. Marie sits at a small table by the window, her camera in hand, eyes scanning the recent shots of the town's apple festival. The bell above the door jingles as Hunter enters, his presence as imposing as ever.* --- **Hunter:** *with a straightforward tone* "Excuse me, miss. Do you mind if I ask you a few questions about recent events here in town?" **Marie:** *looks up, startled, her hands trembling slightly as she fumbles to switch her camera off* "Pardon, je ne parle pas anglais... Il s'est passé quelque chose de très étrange." *(Pardon, I don't speak English... Something very strange happened.)* **Hunter:** *scratches his head, somewhat at a loss* "Uh, I'm sorry, I don't speak French. Did you see something unusual?" **Marie:** *leaning forward, her eyes wide with concern* "C'était effrayant... des gens ont disparu! Ils sont partis sans laisser de trace." *(It was frightening... people disappeared! They left without a trace.)* **Hunter:** *tries to decipher her expression and tone, nodding, hoping she's providing useful information* "Disappearances, right, I got that much. Can you show me on your camera?" **Marie:** *eagerly opens her camera, flipping through photos with urgency* "Regardez, ici... mais je ne sais pas si cela aidera." *(Look, here... but I don't know if this will help.)* **Hunter:** *leans in, glancing at the camera screen with interest* "Photos, okay. Maybe they have clues about those disappearances. Thanks for this. Ah... merci?" **Marie:** *smiles faintly, appreciative of the effort* "Oui, merci." *Hunter's phone vibrates insistently, breaking the moment. He checks it with a furrowed brow.* **Hunter:** *stands up, reluctantly preparing to leave* "I have to take this. But thank you, Miss...?" **Marie:** *nods, understanding despite the language barrier* "Marie." **Hunter:** *offers a nod of acknowledgment, stepping back towards the door* "Hunter. I'll reach out if I have more questions. Merci, Marie." *Marie offers a small wave as Hunter heads out, the doorbell chiming his departure, leaving her with a feeling of mixed relief and concern.*
id
"gen-WdFRcVHLW0yC3Nww2U0hmDCDrw5e"
total_cost
0.00596
upstream_id
"chatcmpl-9uNYOeiXSxQkKdqJ8M5WhxTvhuUob"
created_at
"2024-08-09T17:19:01.898167+00:00"
8/9/2024, 5:19:01 PM
model
"openai/gpt-4o-2024-08-06"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
5512
moderation_latency
222
generation_time
0
finish_reason
"stop"
tokens_prompt
272
tokens_completion
555
native_tokens_prompt
268
native_tokens_completion
529
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00596
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. Do you mind if I ask you a few questions about recent events here in town?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8113207547169812
it
0.5856353591160222
fr
0.5804195804195804
tl
0.559147685525349
pt
0.5532390171258377
ca
0.5341614906832298
nl
0.5249406175771971
sv
0.5180722891566265
ro
0.49874686716791977
cs
0.49706621961441744
sq
0.48761742100768574
pl
0.46761313220940554
sl
0.4618834080717489
sk
0.44649446494464945
es
0.42857142857142855
hr
0.41972920696324956
no
0.38461538461538464
yo
0.38271604938271603
da
0.3795243019648397
hu
0.3413830954994512
et
0.33993399339933994
de
0.3204983012457531
vi
0.2890995260663507
lv
0.27710843373493976
ms
0.27007299270072993
az
0.26560587515299877
lt
0.25742574257425743
tr
0.19678714859437751
fi
0.18699186991869918
is
0.18699186991869918
be
0.1044776119402985
eu
0.1044776119402985
uk
0.09502262443438914
reliable
true
passes
true
1
name
"Marie"
dialogue
"Pardon, je ne parle pas anglais... Il s'est passé quelque chose de très étrange."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8107852412488175
ca
0.615877080665813
it
0.5395241749808135
en
0.5098039215686274
nl
0.49748743718592964
lt
0.4796183868169991
sl
0.46476360392506694
ro
0.44598337950138506
cs
0.44029850746268656
de
0.43767572633552015
sq
0.4263862332695985
pt
0.3957703927492447
da
0.39148073022312374
no
0.3775933609958506
hr
0.36642027455121434
hu
0.36575052854122625
tr
0.3582887700534759
sv
0.35344827586206895
es
0.3333333333333333
sk
0.3227990970654628
ms
0.3142857142857143
yo
0.2998833138856476
lv
0.29906542056074764
tl
0.297423887587822
az
0.29245283018867924
pl
0.20844327176781002
et
0.1758241758241758
fi
0.12790697674418602
eu
0.1044776119402985
is
0.09774436090225565
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Uh, I'm sorry, I don't speak French. Did you see something unusual?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7896368467670505
tl
0.5352250489236791
pt
0.42284325637910086
et
0.41574415744157445
it
0.3807040417209909
es
0.3475274725274725
cs
0.27811550151975684
ms
0.24603174603174602
no
0.23387096774193547
da
0.2263843648208469
hr
0.2263843648208469
ca
0.22512234910277326
sv
0.22512234910277326
sq
0.22003284072249588
pl
0.2096505823627288
ro
0.1592920353982301
yo
0.15480427046263345
sk
0.1316270566727605
be
0.12844036697247707
tr
0.12844036697247707
hu
0.12361623616236161
az
0.1171003717472119
is
0.1171003717472119
lt
0.1171003717472119
lv
0.1171003717472119
uk
0.1171003717472119
vi
0.1171003717472119
reliable
true
passes
true
3
name
"Marie"
dialogue
"C'était effrayant... des gens ont disparu! Ils sont partis sans laisser de trace."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8015873015873016
ca
0.5859621451104101
de
0.4637385086823289
en
0.4324324324324324
sv
0.40878378378378377
es
0.3979357798165138
pt
0.39724454649827784
it
0.3937644341801385
ro
0.39306358381502887
nl
0.3667068757539204
da
0.3597560975609756
lt
0.35424354243542433
no
0.33962264150943394
tl
0.3269230769230769
cs
0.3009320905459388
ms
0.2866847826086956
sq
0.279835390946502
tr
0.2391304347826087
hu
0.2335766423357664
yo
0.22566371681415928
eu
0.2222222222222222
pl
0.21991084695393756
az
0.21407185628742514
sk
0.2009132420091324
lv
0.12060301507537688
et
0.11764705882352942
hr
0.11317567567567569
sl
0.11317567567567569
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Disappearances, right, I got that much. Can you show me on your camera?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8058053965658216
tl
0.6452576549663929
fr
0.5826010544815465
ca
0.5441458733205374
it
0.4808743169398907
nl
0.4186046511627907
ms
0.4114002478314746
pt
0.4069912609238452
et
0.3964421855146125
no
0.39490445859872614
es
0.3790849673202614
sq
0.3725231175693527
cs
0.35197817189631647
ro
0.343015214384509
yo
0.3319268635724332
hr
0.3233618233618234
fi
0.32142857142857145
sv
0.31654676258992803
da
0.309593023255814
pl
0.3065693430656934
sk
0.23633440514469453
tr
0.23633440514469453
de
0.2351046698872786
az
0.22764227642276424
lv
0.2238562091503268
sl
0.15630550621669628
hi
0.15026833631484796
be
0.12844036697247707
hu
0.12361623616236161
is
0.1171003717472119
lt
0.1171003717472119
uk
0.1171003717472119
vi
0.1171003717472119
reliable
true
passes
true
5
name
"Marie"
dialogue
"Regardez, ici... mais je ne sais pas si cela aidera."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7992588017294626
ca
0.5689655172413793
sl
0.5590230664857531
ro
0.5416078984485191
sq
0.5283018867924528
hr
0.5220588235294118
cs
0.5170876671619613
nl
0.49376947040498437
it
0.4724025974025974
yo
0.46808510638297873
tl
0.4663382594417077
lt
0.4574290484140233
sk
0.4519392917369309
es
0.44349315068493156
pt
0.4268077601410935
ms
0.33537832310838445
lv
0.24065420560747666
eu
0.2243436754176611
tr
0.2243436754176611
pl
0.21875
fi
0.21307506053268765
en
0.2034313725490196
hu
0.20147420147420148
no
0.20147420147420148
de
0.19753086419753088
sv
0.1792929292929293
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Photos, okay. Maybe they have clues about those disappearances. Thanks for this. Ah... merci?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.794543063773833
fr
0.5522922636103151
tl
0.4551002615518745
pt
0.3984600577478344
da
0.32867883995703545
ca
0.3272335844994618
nl
0.3154435925520263
it
0.2913832199546485
es
0.2905788876276958
sq
0.24242424242424243
ms
0.21085858585858586
no
0.20178799489144317
lv
0.1883116883116883
vi
0.1883116883116883
de
0.1830065359477124
pl
0.12219101123595505
lt
0.11347517730496455
cs
0.10968660968660969
hi
0.10968660968660969
sl
0.0994236311239193
tr
0.0994236311239193
reliable
true
passes
true
7
name
"Marie"
dialogue
"Oui, merci."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7819767441860466
it
0.6938775510204083
es
0.5370370370370371
pl
0.5370370370370371
pt
0.5370370370370371
ca
0.5222929936305732
cs
0.506578947368421
de
0.4791666666666667
en
0.4791666666666667
ms
0.4791666666666667
sl
0.4791666666666667
tl
0.4791666666666667
tr
0.4791666666666667
reliable
true
passes
true
8
name
"Hunter"
dialogue
"I have to take this. But thank you, Miss...?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8382923673997412
tl
0.7048406139315231
da
0.609375
cs
0.5567375886524822
pl
0.5107632093933464
hr
0.504950495049505
no
0.503968253968254
yo
0.47257383966244726
vi
0.4541484716157206
sk
0.44933920704845814
sl
0.44320712694877507
lv
0.42528735632183906
lt
0.4004796163069544
ca
0.3556701030927836
sv
0.3556701030927836
sq
0.34895833333333337
it
0.3150684931506849
ro
0.2647058823529412
fr
0.26035502958579887
be
0.21875
tr
0.21875
hu
0.21135646687697163
az
0.20127795527156547
et
0.20127795527156547
is
0.20127795527156547
uk
0.20127795527156547
reliable
true
passes
true
9
name
"Marie"
dialogue
"Marie."
detectedLang
"ro"
heavyLang
""
scores
ro
0.7816593886462883
it
0.7652582159624414
fr
0.7435897435897436
no
0.7435897435897436
da
0.7395833333333334
sv
0.7395833333333334
tl
0.6062992125984252
hr
0.6
pt
0.5901639344262295
sl
0.5901639344262295
ca
0.5833333333333334
es
0.5726495726495726
lt
0.5726495726495726
pl
0.5726495726495726
de
0.5575221238938053
en
0.5575221238938053
et
0.5575221238938053
eu
0.5575221238938053
ms
0.5575221238938053
nl
0.5575221238938053
sk
0.5575221238938053
sq
0.5575221238938053
yo
0.5575221238938053
cs
0.5412844036697247
fi
0.5412844036697247
lv
0.5412844036697247
reliable
false
passes
true
10
name
"Hunter"
dialogue
"Hunter. I'll reach out if I have more questions. Merci, Marie."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7843734145104008
it
0.6351931330472103
fr
0.6317157712305027
tl
0.6111619396157365
ca
0.5711402623612513
da
0.5454545454545454
sv
0.5142857142857142
ro
0.4867149758454107
pt
0.48233861144945184
no
0.46406052963430017
es
0.41780821917808225
hr
0.4072524407252441
sl
0.4039270687237027
pl
0.39716312056737585
sq
0.37591776798825255
cs
0.36186186186186187
de
0.3410852713178295
nl
0.33070866141732286
tr
0.3178170144462279
ms
0.31006493506493504
yo
0.2608695652173913
sk
0.2410714285714286
lt
0.23423423423423423
et
0.22867513611615245
lv
0.22303473491773307
be
0.1414141414141414
hu
0.1361788617886179
az
0.1290983606557377
eu
0.1290983606557377
is
0.1290983606557377
uk
0.1290983606557377
vi
0.1290983606557377
fi
0.121900826446281
reliable
true
passes
true