NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2AWVPCTKWWX44BNRA4WDN
Run Details
Model:
GPT-4o, May 13th (temp=0)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
7.3s
Cost:
$0.008610
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet: Scene at the Cafe** *Setting: A quaint cafe in Alborn, MN. The aroma of freshly brewed coffee fills the air. Selvi is seated at a corner table, intently looking at her camera, her face a mix of concentration and worry. Hunter, with his no-nonsense demeanor, approaches her table.* --- **Hunter:** "Excuse me, miss. I'm Detective Hunter. I need to ask you about a recent event in town. Have you seen anything unusual?" **Selvi:** "मुझे नहीं पता कि आप क्या कह रहे हैं।" (I don't know what you are saying.) **Hunter:** *frowning* "I'm sorry, I don't understand. Do you speak English?" **Selvi:** *shaking her head, visibly troubled* "नहीं, मैं अंग्रेजी नहीं बोलती।" (No, I don't speak English.) **Hunter:** *sighing, trying to think of a way to communicate* "Okay, um... event? Disappearances?" *mimes someone vanishing* **Selvi:** *eyes widening, nodding vigorously* "हाँ, हाँ! मैंने कुछ देखा।" (Yes, yes! I saw something.) **Hunter:** *looking more confused* "Alright, let's try this... camera?" *points to her camera* "Pictures?" **Selvi:** *understanding, she quickly turns on her camera and shows him a series of photos, pointing to one in particular* "यहाँ, यहाँ।" (Here, here.) **Hunter:** *leaning in to look at the photo, trying to make sense of it* "This... this is helpful. Thank you." **Selvi:** *nodding, still looking anxious* "कृपया, मुझे मदद चाहिए।" (Please, I need help.) **Hunter:** *looking at her with concern* "I wish I could understand you. I'll get someone who can translate." *Hunter's phone rings. He answers it, his expression turning serious.* **Hunter:** "I have to go. Stay here, okay? I'll be back." **Selvi:** *nodding, though still looking worried* "ठीक है।" (Okay.) *Hunter leaves the cafe, glancing back at Selvi one last time before hurrying out.* --- *End of Scene*
id
"gen-O4m4X0oYGOy4hXUDPFNzEqgJQPFZ"
total_cost
0.00861
upstream_id
"chatcmpl-9uNXFIcFnLlh3tUe2nRC9i6jypxU4"
created_at
"2024-08-09T17:17:51.653027+00:00"
8/9/2024, 5:17:51 PM
model
"openai/gpt-4o-2024-05-13"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
6510
moderation_latency
579
generation_time
0
finish_reason
"stop"
tokens_prompt
273
tokens_completion
577
native_tokens_prompt
273
native_tokens_completion
483
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00861
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. I'm Detective Hunter. I need to ask you about a recent event in town. Have you seen anything unusual?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8053741838272226
tl
0.5905969360802958
ro
0.5395127748068924
it
0.5245398773006136
da
0.5028864656831302
fr
0.48641484426772696
ca
0.48470744680851063
pt
0.4826435246995994
nl
0.4774106540795684
sv
0.46808510638297873
cs
0.4610570236439499
no
0.45306986591390264
sl
0.45306986591390264
es
0.448005698005698
sq
0.4388124547429399
pl
0.40108191653786707
hr
0.39072327044025157
sk
0.3844320889594917
et
0.375
de
0.3699186991869919
ms
0.3600330305532618
yo
0.35737976782752895
lt
0.28702851885924563
lv
0.2475728155339806
hu
0.22344689378757512
az
0.21875
tr
0.20998980632008157
fi
0.15115005476451257
is
0.15115005476451257
vi
0.14647577092511013
be
0.08284023668639054
eu
0.08284023668639054
uk
0.07517899761336516
reliable
true
passes
true
1
name
"Selvi"
dialogue
"मुझे नहीं पता कि आप क्या कह रहे हैं।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8850750842782715
mr
0.8782072101331602
reliable
true
passes
true
2
name
"Hunter"
dialogue
"I'm sorry, I don't understand. Do you speak English?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7994652406417112
no
0.5828698553948832
da
0.5772266065388951
tl
0.5300751879699248
sv
0.5288944723618091
cs
0.5161290322580645
pl
0.4806094182825485
hr
0.4688385269121813
sk
0.438622754491018
de
0.4140625
pt
0.40570522979397783
sq
0.4009584664536741
ro
0.3862520458265139
it
0.3644067796610169
nl
0.3218806509945751
vi
0.3169398907103825
yo
0.3055555555555556
ca
0.2690058479532163
is
0.267578125
sl
0.24242424242424243
ms
0.1610738255033557
be
0.15730337078651685
tr
0.15730337078651685
hu
0.1515837104072398
az
0.14383561643835618
et
0.14383561643835618
lt
0.14383561643835618
lv
0.14383561643835618
uk
0.14383561643835618
reliable
true
passes
true
3
name
"Selvi"
dialogue
"नहीं, मैं अंग्रेजी नहीं बोलती।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.88558352402746
mr
0.8801534036433365
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Okay, um... event? Disappearances?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7211895910780669
fr
0.6549079754601227
pt
0.6428571428571429
tl
0.5967741935483871
ca
0.5033112582781457
de
0.471830985915493
nl
0.460431654676259
it
0.4289340101522843
es
0.42455242966751916
is
0.4094488188976378
ms
0.303405572755418
ro
0.27884615384615385
lv
0.2474916387959866
da
0.23469387755102045
no
0.23469387755102045
sv
0.23469387755102045
reliable
true
passes
true
5
name
"Selvi"
dialogue
"हाँ, हाँ! मैंने कुछ देखा।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8956469886702445
mr
0.8900062853551226
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Pictures?"
detectedLang
"en"
heavyLang
"fr"
scores
en
0.7844827586206896
fr
0.7191011235955056
ca
0.6938775510204083
ro
0.669603524229075
tl
0.669603524229075
et
0.5664739884393063
lv
0.5253164556962026
da
0.4897959183673469
de
0.4897959183673469
nl
0.4897959183673469
no
0.4897959183673469
sv
0.4897959183673469
reliable
true
passes
true
7
name
"Selvi"
dialogue
"यहाँ, यहाँ।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8786407766990292
mr
0.855072463768116
reliable
false
passes
true
8
name
"Hunter"
dialogue
"This... this is helpful. Thank you."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8159831756046267
nl
0.5853080568720379
tl
0.5042492917847025
hu
0.46808510638297873
vi
0.453125
yo
0.4067796610169492
fi
0.3613138686131387
is
0.3445692883895131
cs
0.308300395256917
reliable
true
passes
true
9
name
"Selvi"
dialogue
"कृपया, मुझे मदद चाहिए।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8817204301075269
mr
0.8711340206185567
reliable
true
passes
true
10
name
"Hunter"
dialogue
"I wish I could understand you. I'll get someone who can translate."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8120300751879699
no
0.5771144278606966
da
0.5420258620689655
sv
0.4897959183673469
tl
0.45928753180661575
ro
0.4378306878306878
cs
0.43408788282290284
ca
0.40559440559440557
hr
0.39458689458689455
pl
0.39198855507868385
de
0.3795620437956204
it
0.37315634218289084
nl
0.36186186186186187
is
0.3580060422960725
sk
0.34514637904468415
tr
0.33070866141732286
sq
0.3264659270998415
az
0.3189102564102564
fr
0.3111831442463533
pt
0.27101200686106347
es
0.2672413793103448
vi
0.2621527777777778
lt
0.23423423423423423
lv
0.23423423423423423
et
0.22303473491773307
yo
0.169921875
be
0.1414141414141414
eu
0.1361788617886179
hu
0.1361788617886179
uk
0.1290983606557377
ms
0.121900826446281
sl
0.121900826446281
reliable
true
passes
true
11
name
"Hunter"
dialogue
"I have to go. Stay here, okay? I'll be back."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8055987558320373
tl
0.6402877697841727
yo
0.62178517397882
da
0.609375
pl
0.5961227786752827
lt
0.5274102079395085
hr
0.504950495049505
no
0.503968253968254
cs
0.48559670781893005
sq
0.4834710743801653
sk
0.44933920704845814
lv
0.42528735632183906
hu
0.423963133640553
ca
0.3556701030927836
sv
0.3556701030927836
sl
0.32614555256064687
it
0.3150684931506849
ro
0.2647058823529412
be
0.21875
tr
0.21875
az
0.20127795527156547
et
0.20127795527156547
is
0.20127795527156547
uk
0.20127795527156547
vi
0.20127795527156547
reliable
true
passes
true
12
name
"Selvi"
dialogue
"ठीक है।"
detectedLang
"mr"
heavyLang
"hi"
scores
mr
0.8933143669985776
hi
0.8923959827833573
reliable
true
passes
true