NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2GTVKAVTW00VAPA55TD37
Run Details
Model:
GPT-4o, Aug. 6th (temp=1)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
5.7s
Cost:
$0.004553
Provider:
OpenAI
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet: Cafe Scene** *Selvi is sitting at a small outdoor table at the Alborn cafe, fiddling nervously with her camera. Hunter, in his typical determined manner, walks up to her table.* Hunter: "Excuse me, miss. I heard you might have been at the park when the last disappearance happened. Can I ask you a few questions?" *Selvi looks up, her eyes wide with distress. She glances at her camera, then back at Hunter.* Selvi: "मैंने देखा कुछ अजीब। यह बहुत डरावना था।" (I saw something strange. It was very frightening.) *Hunter shifts his weight, trying to maintain his authoritative presence, but looks visibly puzzled.* Hunter: "I'm sorry, I don't understand. Do you speak any English?" *Selvi shakes her head, managing a half-hearted attempt to communicate.* Selvi: "कमरे में लोग थे लेकिन वो अचानक गायब हो गए।" (There were people in the room, but they suddenly disappeared.) *Hunter scratches his head, clearly at a loss.* Hunter: "I'm not sure what you're saying. Something about people disappearing?" *Selvi nods vigorously, trying to convey urgency through her expressions and gestures.* Selvi: "हां, हां! लोग गायब!" (Yes, yes! People disappeared!) *Hunter opens his mouth to ask another question, but his phone rings, interrupting him. He pulls it out, glancing at the number with a sigh.* Hunter: "I need to take this call, but I'll be back. If you remember anything or find someone to translate, please let me know." *Selvi nods, looking a bit relieved that someone is taking her seriously despite the language barrier.* *Hunter steps away, mouthing a quick apology as he answers his phone, his frustration visible as he walks towards his car.*
id
"gen-7ERhNlus3SkeVomx8GgplNeDSPEk"
total_cost
0.0045525
upstream_id
"chatcmpl-9uNaN7vLkgLpg11ExG3oDMHqugjum"
created_at
"2024-08-09T17:21:04.606352+00:00"
8/9/2024, 5:21:04 PM
model
"openai/gpt-4o-2024-08-06"
app_id
182717
streamed
false
cancelled
false
provider_name
"OpenAI"
latency
5112
moderation_latency
420
generation_time
0
finish_reason
"stop"
tokens_prompt
273
tokens_completion
462
native_tokens_prompt
273
native_tokens_completion
387
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0045525
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. I heard you might have been at the park when the last disappearance happened. Can I ask you a few questions?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8129374849143134
tl
0.6176615688209176
fr
0.5328511151295962
da
0.49113591595535133
it
0.4727891156462585
no
0.47026657552973344
nl
0.45879888268156427
ca
0.38880126182965297
sq
0.38783570300157977
pt
0.37297734627831713
yo
0.36527436527436524
cs
0.33016421780466726
es
0.32491289198606277
ms
0.2941712204007286
hr
0.2857142857142857
et
0.2777260018639329
sv
0.2647058823529412
sk
0.2519305019305019
sl
0.23720472440944884
pl
0.22962226640159045
ro
0.2280876494023904
tr
0.22422422422422422
hu
0.22344689378757512
az
0.2131979695431472
de
0.2083758937691522
lt
0.15760869565217392
fi
0.15115005476451257
is
0.15115005476451257
lv
0.14647577092511013
vi
0.14647577092511013
be
0.08284023668639054
eu
0.08284023668639054
uk
0.07517899761336516
reliable
true
passes
true
1
name
"Selvi"
dialogue
"मैंने देखा कुछ अजीब। यह बहुत डरावना था।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8809523809523809
mr
0.8760438413361169
reliable
true
passes
true
2
name
"Hunter"
dialogue
"I'm sorry, I don't understand. Do you speak any English?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8033431661750245
no
0.5670995670995671
da
0.5614035087719298
tl
0.5139732685297691
sv
0.51278928136419
cs
0.5
pl
0.464524765729585
hr
0.4528043775649795
sk
0.4227994227994228
ca
0.4038748137108793
de
0.39849624060150374
pt
0.3902439024390244
sq
0.38556067588325654
ro
0.3710691823899371
it
0.34959349593495936
nl
0.3079584775086505
vi
0.30313588850174217
yo
0.2920353982300885
is
0.25512104283054005
sl
0.23076923076923078
ms
0.15254237288135594
be
0.14893617021276595
tr
0.14893617021276595
hu
0.14346895074946467
az
0.13606911447084233
et
0.13606911447084233
lt
0.13606911447084233
lv
0.13606911447084233
uk
0.13606911447084233
reliable
true
passes
true
3
name
"Selvi"
dialogue
"कमरे में लोग थे लेकिन वो अचानक गायब हो गए।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8838914690784649
mr
0.876205368777691
reliable
true
passes
true
4
name
"Hunter"
dialogue
"I'm not sure what you're saying. Something about people disappearing?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.817658349328215
tl
0.5489078822412157
ms
0.3606998654104979
nl
0.31948424068767906
tr
0.2725880551301685
da
0.2030201342281879
no
0.20168067226890754
sv
0.1935483870967742
it
0.15480427046263345
de
0.14721723518850988
es
0.1410488245931284
pt
0.1410488245931284
fr
0.1394927536231884
reliable
true
passes
true
5
name
"Selvi"
dialogue
"हां, हां! लोग गायब!"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8837209302325582
mr
0.8828491096532334
reliable
true
passes
true
6
name
"Hunter"
dialogue
"I need to take this call, but I'll be back. If you remember anything or find someone to translate, please let me know."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8200143988480921
tl
0.6067121132669114
no
0.46619217081850534
da
0.4615936826992103
sl
0.43566591422121903
yo
0.410840534171249
cs
0.39271255060728744
sq
0.37707641196013286
sv
0.3734335839598998
fr
0.36815501263689976
et
0.36655405405405406
sk
0.34497816593886466
ms
0.3444055944055944
hr
0.3438320209973753
hu
0.32493249324932494
ro
0.32249322493224936
lt
0.31569343065693434
it
0.30747922437673125
nl
0.30747922437673125
pl
0.30619796484736356
es
0.25521350546176763
lv
0.251497005988024
pt
0.24698795180722888
de
0.2291880781089414
is
0.22440537745604963
ca
0.22118380062305293
vi
0.16759156492785796
tr
0.1506228765571914
az
0.13990825688073394
fi
0.09090909090909091
be
0.08536585365853659
eu
0.08200734394124846
uk
0.07749077490774908
reliable
true
passes
true