NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2DY4VY3CTW1CNQ82N45Q4
Run Details
Model:
GPT-4o, May 13th (temp=0)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
7.6s
Cost:
$0.010305
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
92.31%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet: Scene at the Cafe** --- **[INT. CAFE - DAY]** *Selvi is sitting at a small table by the window, her camera in hand. She is reviewing the photos she took earlier. The cafe is cozy, with the aroma of freshly brewed coffee filling the air. Hunter, dressed in his usual detective attire, enters the cafe and spots Selvi. He approaches her table with a determined look.* --- **Hunter:** "Excuse me, miss. Do you mind if I ask you a few questions?" **Selvi:** *[Looks up, startled, and responds in Hindi]* "क्या आप मुझसे बात कर रहे हैं? मुझे अंग्रेजी नहीं आती।" (Are you talking to me? I don't know English.) **Hunter:** *[Furrows his brow, clearly not understanding]* "I'm sorry, I don't understand. Do you speak English?" **Selvi:** *[Shakes her head, looking troubled]* "नहीं, मुझे अंग्रेजी नहीं आती।" (No, I don't know English.) **Hunter:** *[Sighs, trying to think of a way to communicate]* "Okay, um... event? Town? Disappearances?" *[He gestures with his hands, trying to convey his message]* **Selvi:** *[Her eyes widen, and she responds in Hindi, her voice shaking]* "हाँ, मैंने कुछ अजीब देखा था।" (Yes, I saw something strange.) **Hunter:** *[Looks even more confused, but tries to stay patient]* "Strange? What did you see?" *[He points to his eyes and then around the cafe]* **Selvi:** *[Nods, trying to explain with gestures and broken English]* "Camera... photo... strange... man..." *[She points to her camera and then mimics taking a photo]* **Hunter:** *[Nods, understanding a little]* "Photo? Can I see?" *[He points to her camera]* **Selvi:** *[Hesitates, then hands over her camera, showing him a photo of a suspicious-looking man]* "यह आदमी... अजीब था।" (This man... was strange.) **Hunter:** *[Looks at the photo, then back at Selvi]* "Thank you. This is helpful." *[He tries to smile reassuringly]* "Stay safe." *Hunter's phone rings, and he answers it, his expression turning serious.* **Hunter:** "Hunter here. Yeah, I'll be right there." *[He looks at Selvi apologetically]* "I have to go. Thank you for your help." **Selvi:** *[Nods, still looking shaken]* "ठीक है।" (Okay.) *Hunter leaves the cafe, and Selvi watches him go, still holding her camera tightly.* --- **[END SCENE]**
id
"gen-sVZstlRjfycF0gBrTCYUJMq1Btxj"
total_cost
0.010305
upstream_id
"chatcmpl-9uNYqrcFZ8m0FWWFgQ09iiLSiA39m"
created_at
"2024-08-09T17:19:31.664952+00:00"
8/9/2024, 5:19:31 PM
model
"openai/gpt-4o-2024-05-13"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
7093
moderation_latency
287
generation_time
0
finish_reason
"stop"
tokens_prompt
273
tokens_completion
681
native_tokens_prompt
273
native_tokens_completion
596
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.010305
Evaluation details
Result
Evaluator
Details
Meta Data
92.31%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. Do you mind if I ask you a few questions?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7949699289229087
pt
0.5980707395498392
fr
0.5972073039742213
it
0.5941558441558442
sq
0.5567375886524822
cs
0.5443499392466585
tl
0.5341614906832298
ca
0.5259165613147914
pl
0.4993324432576769
hr
0.4973190348525469
sk
0.4710860366713681
hu
0.4533527696793003
es
0.45255474452554745
et
0.4517543859649123
no
0.44933920704845814
sv
0.44526627218934917
sl
0.4427934621099554
yo
0.43095599393019723
nl
0.4038155802861685
vi
0.39418416801292405
ro
0.37913907284768217
da
0.36006825938566556
ms
0.2816091954022989
tr
0.2816091954022989
lt
0.27884615384615385
fi
0.2690058479532163
is
0.2690058479532163
az
0.2618110236220473
lv
0.2618110236220473
be
0.15730337078651685
eu
0.15730337078651685
de
0.1515837104072398
uk
0.14383561643835618
reliable
true
passes
true
1
name
"Selvi"
dialogue
"क्या आप मुझसे बात कर रहे हैं? मुझे अंग्रेजी नहीं आती।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8800174520069808
mr
0.8770949720670391
reliable
true
passes
true
2
name
"Hunter"
dialogue
"I'm sorry, I don't understand. Do you speak English?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7994652406417112
no
0.5828698553948832
da
0.5772266065388951
tl
0.5300751879699248
sv
0.5288944723618091
cs
0.5161290322580645
pl
0.4806094182825485
hr
0.4688385269121813
sk
0.438622754491018
de
0.4140625
pt
0.40570522979397783
sq
0.4009584664536741
ro
0.3862520458265139
it
0.3644067796610169
nl
0.3218806509945751
vi
0.3169398907103825
yo
0.3055555555555556
ca
0.2690058479532163
is
0.267578125
sl
0.24242424242424243
ms
0.1610738255033557
be
0.15730337078651685
tr
0.15730337078651685
hu
0.1515837104072398
az
0.14383561643835618
et
0.14383561643835618
lt
0.14383561643835618
lv
0.14383561643835618
uk
0.14383561643835618
reliable
true
passes
true
3
name
"Selvi"
dialogue
"नहीं, मुझे अंग्रेजी नहीं आती।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8822421102213849
mr
0.8814604077761973
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Okay, um... event? Town? Disappearances?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7365648050579557
fr
0.6307237813884786
pt
0.6183206106870229
tl
0.5711835334476844
ca
0.4769874476987448
de
0.44567627494456763
nl
0.43438914027149317
it
0.4033412887828162
es
0.39903846153846156
is
0.3842364532019704
ms
0.2816091954022989
ro
0.258160237388724
lv
0.22839506172839508
da
0.2163009404388715
no
0.2163009404388715
sv
0.2163009404388715
reliable
true
passes
true
5
name
"Selvi"
dialogue
"हाँ, मैंने कुछ अजीब देखा था।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8891129032258065
mr
0.8853211009174312
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Strange? What did you see?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.830316742081448
et
0.604221635883905
tl
0.5844875346260388
da
0.5664739884393063
no
0.5588235294117647
de
0.5327102803738317
nl
0.5253164556962026
cs
0.5176848874598071
fr
0.5
hr
0.3723849372384937
sk
0.3697478991596639
sl
0.3697478991596639
it
0.3562231759656652
pl
0.34210526315789475
ro
0.3333333333333333
sv
0.32432432432432434
lv
0.3181818181818182
is
0.3087557603686636
lt
0.3087557603686636
ms
0.29577464788732394
az
0.2822966507177033
sq
0.2822966507177033
tr
0.2822966507177033
reliable
true
passes
true
7
name
"Selvi"
dialogue
"Camera... photo... strange... man..."
detectedLang
"en"
heavyLang
"fr"
scores
en
0.7448979591836735
fr
0.7038917089678511
tl
0.684115523465704
de
0.6722846441947566
nl
0.6615087040618954
da
0.65
no
0.6187363834422658
sv
0.6162280701754387
it
0.5793269230769231
ro
0.571078431372549
lv
0.5042492917847025
cs
0.4942196531791907
lt
0.4791666666666667
ms
0.43548387096774194
ca
0.36823104693140796
es
0.3613138686131387
hr
0.3371212121212121
sk
0.3346007604562738
sl
0.3346007604562738
pl
0.308300395256917
hi
0.3055555555555556
is
0.2768595041322314
az
0.25213675213675213
et
0.25213675213675213
sq
0.25213675213675213
tr
0.25213675213675213
reliable
true
passes
false
8
name
"Hunter"
dialogue
"Photo? Can I see?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.810318664643399
et
0.6508379888268158
ca
0.6323529411764706
cs
0.6069182389937107
fr
0.6031746031746031
tl
0.5487364620938628
tr
0.5404411764705882
no
0.5370370370370371
az
0.5283018867924528
da
0.5265151515151516
hr
0.5265151515151516
sv
0.5247148288973384
sq
0.5173745173745175
pl
0.50199203187251
it
0.4791666666666667
ro
0.4186046511627907
yo
0.410377358490566
hi
0.3811881188118812
sk
0.3654822335025381
be
0.358974358974359
hu
0.34895833333333337
is
0.33510638297872336
lt
0.33510638297872336
lv
0.33510638297872336
uk
0.33510638297872336
vi
0.33510638297872336
reliable
true
passes
true
9
name
"Selvi"
dialogue
"यह आदमी... अजीब था।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8732394366197184
mr
0.8691099476439791
reliable
true
passes
true
10
name
"Hunter"
dialogue
"Stay safe."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8076923076923077
reliable
false
passes
true
11
name
"Hunter"
dialogue
"I have to go. Thank you for your help."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8371335504885994
da
0.6871088861076344
tl
0.6635262449528937
no
0.622356495468278
pl
0.5961227786752827
cs
0.5567375886524822
yo
0.5462794918330308
hr
0.504950495049505
vi
0.4541484716157206
sk
0.44933920704845814
lv
0.42528735632183906
lt
0.4004796163069544
ca
0.3556701030927836
sv
0.3556701030927836
sq
0.34895833333333337
sl
0.32614555256064687
it
0.3150684931506849
ro
0.2647058823529412
pt
0.25595238095238093
ms
0.2378048780487805
be
0.21875
tr
0.21875
hu
0.21135646687697163
az
0.20127795527156547
et
0.20127795527156547
is
0.20127795527156547
uk
0.20127795527156547
reliable
true
passes
true
12
name
"Selvi"
dialogue
"ठीक है।"
detectedLang
"mr"
heavyLang
"hi"
scores
mr
0.8933143669985776
hi
0.8923959827833573
reliable
true
passes
true