NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-09-20-01J881E3RE03CPQD1WYHQ1XCNC
Run Details
Model:
Cohere Command R+ (Aug. 2024)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
14.9s
Cost:
$0.007783
Provider:
Cohere
Evaluation summary
Score
Evaluator
Details Link
91.67%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
# Scene: A Cozy Mystery in Alborn, MN --- **Marie**: (sitting alone at an outdoor café table, her DSLR camera resting on the table in front of her, she flips through the photos she took earlier, a concerned expression on her face) **Hunter**: (walking up to Marie, his detective's eyes narrowing as he approaches the young woman) Excuse me, mademoiselle. I couldn't help but notice your reaction when looking at those photos. Is something bothering you? (his voice is deep and authoritative, reflecting his marine background) **Marie**: (startled, she looks up at Hunter, and speaks in French, her voice shaking) Je... je ne comprends pas. Que se passe-t-il dans cette ville ? Il y a quelque chose d'étrange. (Translation: I... I don't understand. What's happening in this town? There's something eerie going on.) **Hunter**: (his face falls as he realizes she's speaking French, a language he doesn't know) Oh... um, I... (He trails off, feeling at a loss for words, then points to himself and shrugs, indicating he doesn't understand.) **Marie**: (noticing his confusion, she tries again, slowly and articulating each word) Un mystérieux... dis-parition... ici. (She points to her camera, indicating something about the photos.) **Hunter**: (understanding the word 'disappearance', his detective instincts kick in) Are you talking about the recent disappearances around town? Someone has gone missing again? (He leans in, his voice softening a bit as he tries to communicate.) **Marie**: (nodding fervently) Oui, oui! J'ai vu quelque chose. Ces photos... je les ai prises hier. (She scrolls to a particular image and shows him a photo of the town's park, then switches to show multiple similar photos.) **Hunter**: (studying the photos, he notices a subtle difference in each one) These are the same spot but at different times. What did you see? **Marie**: (she points to a vague figure in the background of the first photo) Là-bas. Une personne s'est envolée. Juste... disparu. (Her eyes fill with tears, and she whispers:) Je n'ai pas pu aider. **Hunter**: (his face pales, and he glances around the café, then leans in closer, his voice hushed) Tell me, when did this happen? Was there any sign of a struggle? **Marie** (continues to explain what she witnessed through her camera, her voice shaky but determined, as Hunter listens intently, taking mental notes.) --- (Hunter's phone interrupts their conversation, and he steps away to answer it.) **Hunter**: (into the phone) Detective Hunter here. ... I see. I'll be right there. (He turns back to Marie, his expression apologetic.) I have to take this call, but we'll continue this soon. (He gestures for her to stay put.) **Marie**: (her eyes question him, but she manages a small nod.) --- **Marie**: ... **Hunter**: ... (He walks swiftly away to address the new development in his investigation.)
id
"gen-N1IOYCLFGpBTTRr9snc985wBkTGl"
total_cost
0.0077825
upstream_id
"e700d17a-fdac-4e91-ab2c-7d9e9c0db53f"
created_at
"2024-09-20T15:40:14.745995+00:00"
9/20/2024, 3:40:14 PM
model
"cohere/command-r-plus-08-2024"
app_id
182717
streamed
true
cancelled
false
provider_name
"Cohere"
latency
62
moderation_latency
(null)
generation_time
14533
finish_reason
"COMPLETE"
tokens_prompt
272
tokens_completion
685
native_tokens_prompt
289
native_tokens_completion
706
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0077825
Evaluation details
Result
Evaluator
Details
Meta Data
91.67%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, mademoiselle. I couldn't help but notice your reaction when looking at those photos. Is something bothering you?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7846948818897638
tl
0.6031746031746031
fr
0.5787193066923447
no
0.4696969696969697
nl
0.468408262454435
da
0.4254760341431385
de
0.38939288206559663
pt
0.35944363103953153
sv
0.3289877300613497
es
0.3264049268668206
et
0.32327919566898683
it
0.3227554179566564
fi
0.31156569630212433
sq
0.30389817024661897
ro
0.2857142857142857
ms
0.27506213753106873
yo
0.25531914893617025
ca
0.2450388265746333
cs
0.23380035026269705
lv
0.22429078014184398
is
0.2052679382379655
hr
0.20309653916211293
hu
0.20164233576642335
tr
0.16267942583732056
sk
0.14299706170421156
lt
0.1404715127701375
pl
0.12587412587412586
sl
0.09138110072689512
hi
0.08088235294117647
zh
0.08088235294117647
be
0.07407407407407407
az
0.06716417910447761
uk
0.06716417910447761
vi
0.06716417910447761
reliable
true
passes
true
1
name
"Marie"
dialogue
"Je... je ne comprends pas. Que se passe-t-il dans cette ville ? Il y a quelque chose d'étrange."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8176206509539842
it
0.6520342612419701
ca
0.6292070735881347
en
0.6014714898835071
da
0.5909376966645689
es
0.5787427090084252
no
0.5489243580846634
de
0.543859649122807
nl
0.5357142857142857
pt
0.5327102803738317
hr
0.4749596122778676
sl
0.4724025974025974
ro
0.4645799011532125
cs
0.4610281923714759
sq
0.4610281923714759
tl
0.4472789115646259
sv
0.4468085106382979
hu
0.4263018534863195
yo
0.41016333938294003
et
0.406392694063927
fi
0.406392694063927
lt
0.39981532779316714
tr
0.38095238095238093
sk
0.3394308943089431
pl
0.31650893796004204
is
0.29577464788732394
az
0.27293064876957496
lv
0.2630385487528345
vi
0.26220204313280365
ms
0.2459396751740139
eu
0.09722222222222222
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Oh... um, I..."
detectedLang
"is"
heavyLang
"pt"
scores
is
0.7448979591836735
pt
0.6724890829694322
no
0.6590909090909091
da
0.6495327102803737
hr
0.6495327102803737
ca
0.647887323943662
sv
0.647887323943662
sq
0.6411483253588517
de
0.6376811594202898
pl
0.6268656716417911
nl
0.625
it
0.6052631578947368
cs
0.5833333333333334
en
0.5738636363636364
ro
0.5454545454545454
yo
0.5370370370370371
tl
0.5
sk
0.4897959183673469
be
0.4827586206896552
tr
0.4827586206896552
hu
0.471830985915493
az
0.45652173913043476
et
0.45652173913043476
lt
0.45652173913043476
lv
0.45652173913043476
uk
0.45652173913043476
vi
0.45652173913043476
reliable
true
passes
false
3
name
"Marie"
dialogue
"Un mystérieux... dis-parition... ici."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7925311203319502
en
0.5614035087719298
yo
0.5309568480300189
de
0.5164410058027079
da
0.494949494949495
sv
0.4897959183673469
it
0.46351931330472107
ro
0.4553376906318083
tl
0.4369369369369369
lv
0.4004796163069544
ms
0.3961352657004831
ca
0.3796526054590571
es
0.3796526054590571
fi
0.37810945273631835
nl
0.35233160621761656
no
0.35233160621761656
pt
0.34895833333333337
hu
0.2753623188405797
zh
0.23547400611620795
cs
0.23076923076923078
sk
0.23076923076923078
et
0.21135646687697163
sq
0.21135646687697163
hr
0.19093851132686085
sl
0.19093851132686085
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Are you talking about the recent disappearances around town? Someone has gone missing again?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.825321408608161
nl
0.5713305898491083
fr
0.5214395099540582
tl
0.5143745143745143
pt
0.4292237442922375
ca
0.42502299908003677
sv
0.3978805394990366
da
0.3272335844994618
ro
0.3146929824561403
ms
0.31394072447859495
no
0.296962879640045
es
0.2690058479532163
cs
0.26297169811320753
is
0.26297169811320753
it
0.26210153482880755
de
0.19562419562419564
sk
0.1935483870967742
yo
0.14617486338797814
fi
0.11095305832147938
sq
0.11095305832147938
lv
0.10586552217453504
pl
0.0994236311239193
az
0.09420289855072464
et
0.09420289855072464
hr
0.09156976744186046
lt
0.09156976744186046
sl
0.09156976744186046
reliable
true
passes
true
5
name
"Marie"
dialogue
"Oui, oui! J'ai vu quelque chose. Ces photos... je les ai prises hier."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8102466793168881
en
0.5824634655532359
de
0.46236559139784944
nl
0.42857142857142855
sl
0.40476190476190477
hr
0.39759036144578314
es
0.38366718027734975
it
0.37888198757763975
ca
0.36507936507936506
sk
0.36
sq
0.35379644588045234
lt
0.3162393162393162
no
0.29577464788732394
tl
0.2895204262877442
yo
0.2895204262877442
cs
0.2687385740402194
pt
0.2687385740402194
ro
0.2172211350293542
et
0.20948616600790515
vi
0.20948616600790515
pl
0.18533604887983707
da
0.17355371900826447
sv
0.17355371900826447
hi
0.16142557651991615
lv
0.15074309978768577
reliable
true
passes
true
6
name
"Hunter"
dialogue
"These are the same spot but at different times. What did you see?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8324793062672446
tl
0.6160794941282746
no
0.5473908413205538
fr
0.5230078563411897
da
0.4487678339818417
it
0.40308988764044945
ca
0.3675595238095238
sq
0.36186186186186187
ro
0.3531202435312024
et
0.2857142857142857
nl
0.2722602739726028
hr
0.2582897033158813
cs
0.24911660777385158
yo
0.20112781954887218
ms
0.1550695825049702
sv
0.1550695825049702
pt
0.1533864541832669
de
0.1290983606557377
es
0.1290983606557377
lt
0.1290983606557377
lv
0.1290983606557377
pl
0.1290983606557377
sk
0.1290983606557377
sl
0.1290983606557377
reliable
true
passes
true
7
name
"Marie"
dialogue
"Là-bas. Une personne s'est envolée. Juste... disparu. Je n'ai pas pu aider."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8004561003420753
ca
0.4615384615384615
de
0.4587628865979381
pt
0.45764462809917356
nl
0.450261780104712
ms
0.39236111111111116
ro
0.37574316290130794
en
0.3613138686131387
it
0.3462017434620174
cs
0.34375
sk
0.34210526315789475
sq
0.33962264150943394
es
0.33035714285714285
yo
0.3234536082474227
et
0.3208279430789133
hr
0.30463576158940403
sv
0.3
no
0.29624664879356566
da
0.2953020134228188
tl
0.27884615384615385
sl
0.24785100286532946
lt
0.2369186046511628
vi
0.23020527859237538
pl
0.2222222222222222
tr
0.20574886535552195
az
0.19601837672281774
lv
0.125
eu
0.1191275167785235
hu
0.10714285714285714
is
0.10714285714285714
reliable
true
passes
true
8
name
"Hunter"
dialogue
"Tell me, when did this happen? Was there any sign of a struggle?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8260335652885797
de
0.5833333333333334
nl
0.5745745745745746
no
0.49223416965352446
tl
0.4785276073619632
sq
0.46202531645569617
yo
0.42876344086021506
it
0.3902439024390244
ca
0.3876080691642651
fr
0.38583815028901736
is
0.38316400580551524
hu
0.37037037037037035
sl
0.3580060422960725
es
0.3550834597875569
pt
0.3491577335375191
sv
0.3441358024691358
da
0.3410852713178295
ms
0.3264659270998415
ro
0.3178170144462279
cs
0.31006493506493504
hr
0.30213464696223313
et
0.27101200686106347
fi
0.24511545293072823
sk
0.23423423423423423
az
0.2328519855595668
lv
0.2328519855595668
pl
0.19811320754716982
lt
0.16173570019723865
tr
0.1533864541832669
eu
0.1414141414141414
vi
0.1414141414141414
reliable
true
passes
true
9
name
"Hunter"
dialogue
"Detective Hunter here. ... I see. I'll be right there. I have to take this call, but we'll continue this soon."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8137002006305533
tl
0.6058217101273499
fr
0.531700288184438
da
0.4921875
ro
0.48899371069182396
ca
0.47622884770346496
it
0.4698205546492659
no
0.42781690140845074
sq
0.41651705565529623
pt
0.41441441441441446
de
0.39478584729981375
yo
0.385633270321361
es
0.37137330754352027
sl
0.3333333333333333
hr
0.33127572016460904
cs
0.32150313152400833
et
0.3193717277486911
lt
0.30032292787944026
sv
0.29729729729729726
pl
0.2864983534577387
nl
0.2696629213483146
sk
0.2388758782201405
lv
0.2215568862275449
hu
0.22062350119904073
ms
0.17407878017789072
tr
0.16559691912708602
ja
0.13333333333333333
be
0.09722222222222222
az
0.08835904628330996
is
0.08835904628330996
uk
0.08835904628330996
vi
0.08835904628330996
reliable
true
passes
true
10
name
"Marie"
dialogue
"..."
detectedLang
""
heavyLang
""
scores
reliable
false
passes
true
11
name
"Hunter"
dialogue
"..."
detectedLang
""
heavyLang
""
scores
reliable
false
passes
true