NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-09-20-01J881MPPTZZD9WQNEB0WP8W7Z
Run Details
Model:
Cohere Command R+ (Aug. 2024)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
12.0s
Cost:
$0.005285
Provider:
Cohere
Evaluation summary
Score
Evaluator
Details Link
87.50%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Cozy Mystery Scene: The Cafe Encounter** *Voice Sheet:* **Selvi:** [Staring at her camera intently, she whispers in Hindi] "क्या आप केवल समय पर होंगे?" [Translation: Will you only show up in time?] **Hunter:** [Leaning over the table, his tone is curious yet firm] "Miss, I'm Detective Hunter. Are you familiar with the recent incidents at the apple orchard? I was told you might have some information." **Selvi:** [She looks up, her eyes wide, and speaks rapidly in Hindi] "मैं... मैं वहां थी, जब वे आए थे। पता नहीं कहाँ से। लेकिन वे उन लोगों को ले गए।" [Translation: I... I was there when they came. I don't know from where, but they took some of the people.] **Hunter:** [Confusion washes over his face] "Uh, I'm sorry, I didn't catch that. Is there a language barrier here?" **Selvi:** [Realizing the miscommunication, she tries to gesture with her hands] "Sorry, I... no English. I can show you, pictures I took." **Hunter:** "You were at the orchard? And you have photos?" [He sits down across from her, grabbing a chair] **Selvi:** [Nodding eagerly, she starts scrolling through her camera's display] "हाँ, हाँ! ये देखो, वे बस आए थे और..." [As she describes the scene, Hunter notices the intensity in her dark eyes] **Hunter:** [Interrupting her Hindi explanation] "Hold on, I've got a call. Stay put, we'll figure this out." [He answers his phone and walks a few steps away] **Selvi:** [Muttering to herself in Hindi, a mix of relief and worry crosses her face] [The dialogue continues in this tense yet intriguing manner, leaving the reader eager to know what crucial information Selvi possesses.]
id
"gen-jSVoGsWqR3O39UUYqfP8EiWN4kY5"
total_cost
0.005285
upstream_id
"d6374507-1353-47bd-a772-52f0a53f6334"
created_at
"2024-09-20T15:43:47.843848+00:00"
9/20/2024, 3:43:47 PM
model
"cohere/command-r-plus-08-2024"
app_id
182717
streamed
true
cancelled
false
provider_name
"Cohere"
latency
52
moderation_latency
(null)
generation_time
11662
finish_reason
"COMPLETE"
tokens_prompt
273
tokens_completion
499
native_tokens_prompt
294
native_tokens_completion
455
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.005285
Evaluation details
Result
Evaluator
Details
Meta Data
87.50%
Parse dialogue
n/a
dialogue
0
name
"Selvi"
dialogue
"क्या आप केवल समय पर होंगे?"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8832614942528736
mr
0.8810395314787701
reliable
true
passes
true
1
name
"Hunter"
dialogue
"Miss, I'm Detective Hunter. Are you familiar with the recent incidents at the apple orchard? I was told you might have some information."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8059436682191173
tl
0.6646224607129169
fr
0.6501399440223911
da
0.6079749103942652
nl
0.571917808219178
ca
0.5693897637795275
de
0.5533435426237876
ro
0.5478036175710594
no
0.5433194154488518
sv
0.5353159851301116
it
0.5173745173745175
es
0.5165745856353591
pt
0.4948036951501155
sq
0.44620253164556967
cs
0.4083840432724814
yo
0.3754461099214847
hr
0.3484735666418466
sl
0.33862433862433866
sk
0.32432432432432434
lt
0.3169398907103825
pl
0.28687856560717195
lv
0.25658453695836875
ms
0.24111014744145706
az
0.22839506172839508
et
0.21805183199285078
eu
0.1898148148148148
tr
0.18528864059590316
hu
0.12935323383084577
fi
0.08568443051201673
zh
0.08088235294117647
be
0.07407407407407407
is
0.06716417910447761
uk
0.06716417910447761
vi
0.06716417910447761
reliable
true
passes
true
2
name
"Selvi"
dialogue
"मैं... मैं वहां थी, जब वे आए थे। पता नहीं कहाँ से। लेकिन वे उन लोगों को ले गए।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8841698841698842
mr
0.8695084819486734
reliable
true
passes
true
3
name
"Hunter"
dialogue
"Uh, I'm sorry, I didn't catch that. Is there a language barrier here?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8063516653756777
tl
0.5265151515151516
it
0.4929006085192698
de
0.4908350305498982
yo
0.4791666666666667
fr
0.46808510638297873
ro
0.4541484716157206
no
0.43883277216610556
da
0.4375703037120359
sq
0.43438914027149317
hu
0.4298745724059293
ca
0.423963133640553
sv
0.4232987312572088
pt
0.3946731234866829
nl
0.3872549019607844
is
0.3857493857493857
hr
0.3342210386151798
eu
0.3315508021390375
cs
0.32157394843962006
pl
0.3160054719562244
es
0.30362116991643456
lt
0.2937853107344633
et
0.28977272727272724
sk
0.2877492877492877
ms
0.23076923076923078
tr
0.22720247295208654
az
0.21011058451816747
lv
0.21011058451816747
vi
0.21011058451816747
sl
0.15254237288135594
be
0.12280701754385964
fi
0.11190053285968028
uk
0.11190053285968028
reliable
true
passes
true
4
name
"Selvi"
dialogue
"Sorry, I... no English. I can show you, pictures I took."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7967479674796748
tl
0.643640350877193
ca
0.6153846153846154
no
0.5492371705963939
pt
0.4921875
ro
0.49139280125195617
hr
0.4897959183673469
it
0.47580645161290325
da
0.46546052631578944
sv
0.4645799011532125
lv
0.46013289036544847
tr
0.4196428571428571
pl
0.3821292775665399
fr
0.37137330754352027
cs
0.3602362204724409
sk
0.3340163934426229
et
0.33127572016460904
nl
0.3143459915611814
de
0.30703624733475476
az
0.3010752688172043
es
0.29653679653679654
sq
0.2919389978213508
yo
0.2111650485436893
ms
0.181360201511335
be
0.17721518987341772
hu
0.1709183673469388
is
0.16237113402061853
lt
0.16237113402061853
uk
0.16237113402061853
vi
0.16237113402061853
reliable
true
passes
false
5
name
"Hunter"
dialogue
"You were at the orchard? And you have photos?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8292349726775956
tl
0.7175141242937854
da
0.6153846153846154
fr
0.5726495726495726
no
0.5644599303135889
nl
0.4791666666666667
yo
0.4517543859649123
de
0.3686868686868687
es
0.24242424242424243
lt
0.24242424242424243
pt
0.24242424242424243
cs
0.2378048780487805
ms
0.2378048780487805
sq
0.2378048780487805
sv
0.2378048780487805
hi
0.23547400611620795
ca
0.22118380062305293
lv
0.22118380062305293
reliable
true
passes
true
6
name
"Selvi"
dialogue
"हाँ, हाँ! ये देखो, वे बस आए थे और..."
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8841698841698842
mr
0.869394862864606
reliable
true
passes
true
7
name
"Hunter"
dialogue
"Hold on, I've got a call. Stay put, we'll figure this out."
detectedLang
"en"
heavyLang
"en"
scores
en
0.803038829487901
fr
0.5159059474412172
tl
0.5042492917847025
hr
0.4814814814814815
ca
0.4712990936555891
ro
0.4496855345911949
da
0.4195688225538972
et
0.38271604938271603
fi
0.3783303730017762
pt
0.3738819320214669
cs
0.37163375224416517
sk
0.36936936936936937
es
0.36479128856624315
it
0.3566176470588235
yo
0.3566176470588235
pl
0.35064935064935066
hu
0.308300395256917
sq
0.2985971943887776
no
0.2842535787321063
de
0.279835390946502
sl
0.20454545454545453
hi
0.1935483870967742
lt
0.1898148148148148
tr
0.18032786885245902
is
0.1764705882352941
az
0.16666666666666666
eu
0.16666666666666666
lv
0.16666666666666666
nl
0.16666666666666666
vi
0.16666666666666666
ms
0.15254237288135594
sv
0.15254237288135594
reliable
true
passes
true