NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2GXATVAT4VAVZ8XS3S8W5
Run Details
Model:
Mistral Large 2
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
12.6s
Cost:
$0.004072
Evaluation summary
Score
Evaluator
Details Link
54.55%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
Sure, here's how the scene might play out: --- **Selvi:** *(looking intently at her camera, scrolling through photos, she appears troubled)* **Hunter:** *(approaches Selvi, showing his badge)* "Excuse me, miss. I'm Detective Hunter. Mind if I ask you a few questions about the recent event in town?" **Selvi:** *(looks up, startled, and responds in Hindi)* "Kya aap mere paas kyun aaye hain? Main kuch nahi jaanti." *(Translation: "Why are you here? I don't know anything.")* **Hunter:** *(confused)* "I'm sorry, miss. I don't understand. Do you speak English?" **Selvi:** *(shakes her head, looking increasingly troubled)* "Main Hindi bolti hoon. Aapko samajh nahi aata." *(Translation: "I speak Hindi. You don't understand.")* **Hunter:** *(sighs, trying to be patient)* "Look, I just need to know if you saw anything unusual yesterday. Anything at all." **Selvi:** *(trying to convey her message with gestures, points to her camera)* "Main yahan nahi thi. Main sirf photo le rahi thi." *(Translation: "I wasn't here. I was just taking photos.")* **Hunter:** *(nods, understanding her gestures)* "Photos, huh? Mind if I take a look?" **Selvi:** *(hesitates, then hands over her camera)* "Theek hai, lekin main kuch nahi jaanti." *(Translation: "Okay, but I don't know anything.")* **Hunter:** *(scrolls through the photos, nods, then hands the camera back)* "Thanks. If you remember anything, please let me know." **Selvi:** *(takes the camera, looking relieved)* "Theek hai." *(Translation: "Okay.")* **Hunter:** *(receives a call, answers it)* "Hunter here... I see... I'll be right there." *(hangs up)* "I have to go. Thanks for your time." **Selvi:** *(nods, watching Hunter leave, still looking troubled)* --- This scene captures the language barrier and the initial misunderstanding between Selvi and Hunter, while also hinting at the larger mystery at hand.
total_cost
0.004072
Evaluation details
Result
Evaluator
Details
Meta Data
54.55%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Excuse me, miss. I'm Detective Hunter. Mind if I ask you a few questions about the recent event in town?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8095112979506043
fr
0.6153846153846154
tl
0.6122994652406418
it
0.5992260917634052
ro
0.5560318432333129
nl
0.551638837353123
ca
0.5457393483709273
sv
0.5425867507886435
pt
0.5307443365695793
es
0.4641537324464154
no
0.4641537324464154
da
0.4573353293413174
sq
0.4461420932009167
cs
0.4232299124900557
sl
0.41813804173354735
de
0.3855932203389831
yo
0.3561278863232682
pl
0.3462578899909829
hr
0.3444846292947558
ms
0.33118081180811804
sk
0.32683379758588665
hu
0.30019305019305015
et
0.2988394584139265
az
0.23036093418259024
lt
0.2229367631296892
tr
0.22126745435016112
fi
0.15990730011587484
is
0.15990730011587484
lv
0.15501165501165504
vi
0.15501165501165504
be
0.0880503144654088
eu
0.0880503144654088
uk
0.0799492385786802
reliable
true
passes
true
1
name
"Selvi"
dialogue
"Kya aap mere paas kyun aaye hain? Main kuch nahi jaanti."
detectedLang
"eu"
heavyLang
"tl"
scores
eu
0.5024875621890548
da
0.34497816593886466
et
0.30394431554524365
it
0.2665036674816626
en
0.24812030075187969
fr
0.22680412371134018
lt
0.22680412371134018
ms
0.21052631578947367
de
0.20634920634920634
lv
0.2
cs
0.18256130790190736
hr
0.18256130790190736
ro
0.18256130790190736
sk
0.18256130790190736
sl
0.18256130790190736
sq
0.18256130790190736
tl
0.18256130790190736
tr
0.18256130790190736
reliable
false
passes
false
2
name
"Hunter"
dialogue
"I'm sorry, miss. I don't understand. Do you speak English?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7994987468671679
no
0.5670995670995671
tl
0.5666305525460456
da
0.5614035087719298
sv
0.51278928136419
cs
0.5
pl
0.464524765729585
hr
0.4528043775649795
sk
0.4227994227994228
de
0.39849624060150374
pt
0.3902439024390244
sq
0.38556067588325654
ro
0.3710691823899371
it
0.34959349593495936
nl
0.3079584775086505
vi
0.30313588850174217
yo
0.2920353982300885
ca
0.25650557620817843
is
0.25512104283054005
sl
0.23076923076923078
ms
0.15254237288135594
be
0.14893617021276595
tr
0.14893617021276595
hu
0.14346895074946467
az
0.13606911447084233
et
0.13606911447084233
lt
0.13606911447084233
lv
0.13606911447084233
uk
0.13606911447084233
reliable
true
passes
true
3
name
"Selvi"
dialogue
"Main Hindi bolti hoon. Aapko samajh nahi aata."
detectedLang
"tl"
heavyLang
"tl"
scores
tl
0.576271186440678
is
0.574468085106383
it
0.4307400379506641
et
0.39759036144578314
eu
0.3548387096774194
fi
0.25742574257425743
no
0.25
en
0.24812030075187969
lv
0.24433249370277077
fr
0.22680412371134018
sq
0.2248062015503876
da
0.2227979274611399
sv
0.21052631578947367
de
0.20634920634920634
hi
0.20634920634920634
ms
0.19137466307277629
reliable
false
passes
false
4
name
"Hunter"
dialogue
"Look, I just need to know if you saw anything unusual yesterday. Anything at all."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8117320650019818
tl
0.5527306967984934
et
0.5113168724279835
no
0.5077720207253886
da
0.46808510638297873
sv
0.4644870349492672
cs
0.39797211660329534
sq
0.39180537772087065
pl
0.3546195652173913
hr
0.3493150684931507
ms
0.3493150684931507
yo
0.32045779685264664
ca
0.3115942028985507
sk
0.3004418262150221
it
0.2824773413897281
lv
0.2803030303030303
lt
0.2601246105919003
nl
0.2263843648208469
sl
0.2030201342281879
ro
0.1592920353982301
es
0.1394927536231884
pt
0.1394927536231884
de
0.1316270566727605
be
0.12844036697247707
tr
0.12844036697247707
hu
0.12361623616236161
az
0.1171003717472119
is
0.1171003717472119
uk
0.1171003717472119
vi
0.1171003717472119
reliable
true
passes
true
5
name
"Selvi"
dialogue
"Main yahan nahi thi. Main sirf photo le rahi thi."
detectedLang
"fr"
heavyLang
"vi"
scores
fr
0.6408045977011495
en
0.5192307692307693
tl
0.4541484716157206
eu
0.39759036144578314
vi
0.38271604938271603
it
0.3670886075949367
ms
0.3556701030927836
yo
0.32795698924731176
es
0.32249322493224936
sl
0.31880108991825606
ro
0.3131868131868132
hu
0.2816091954022989
cs
0.26035502958579887
fi
0.26035502958579887
de
0.2378048780487805
hi
0.23547400611620795
reliable
true
passes
false
6
name
"Hunter"
dialogue
"Photos, huh? Mind if I take a look?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7916666666666666
tl
0.6439873417721518
ca
0.6052631578947368
hu
0.5802238805970149
sv
0.5722433460076045
fr
0.5706106870229007
hr
0.5273109243697479
it
0.5161290322580645
cs
0.512987012987013
pl
0.506578947368421
ro
0.5044052863436124
lt
0.5
pt
0.4943820224719101
no
0.4886363636363637
et
0.4874715261958998
es
0.4851258581235698
da
0.48394495412844035
sq
0.4779582366589328
yo
0.4779582366589328
lv
0.4755244755244755
sk
0.4730679156908665
sl
0.42748091603053434
tr
0.3951612903225807
nl
0.38524590163934425
de
0.38016528925619836
is
0.38016528925619836
az
0.3715083798882682
vi
0.3715083798882682
hi
0.2549668874172185
be
0.23728813559322032
eu
0.23728813559322032
fi
0.21875
ms
0.21875
uk
0.21875
reliable
true
passes
true
7
name
"Selvi"
dialogue
"Theek hai, lekin main kuch nahi jaanti."
detectedLang
"eu"
heavyLang
"tl"
scores
eu
0.5551601423487544
it
0.4553376906318083
vi
0.3857493857493857
et
0.3438320209973753
nl
0.29577464788732394
en
0.28366762177650434
fi
0.2732558139534884
fr
0.26035502958579887
is
0.26035502958579887
lt
0.26035502958579887
ms
0.24242424242424243
de
0.2378048780487805
lv
0.23076923076923078
cs
0.21135646687697163
hr
0.21135646687697163
ro
0.21135646687697163
sk
0.21135646687697163
sl
0.21135646687697163
sq
0.21135646687697163
tl
0.21135646687697163
tr
0.21135646687697163
reliable
false
passes
false
8
name
"Hunter"
dialogue
"Thanks. If you remember anything, please let me know."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8190154440154441
tl
0.5235069885641677
nl
0.5046235138705416
ms
0.471830985915493
sl
0.46120689655172414
no
0.41222570532915365
sv
0.39999999999999997
sq
0.3980738362760835
da
0.3872549019607844
fr
0.33745583038869253
cs
0.336283185840708
yo
0.33510638297872336
hu
0.3206521739130435
et
0.31569343065693434
de
0.2937853107344633
is
0.29111531190926276
vi
0.27884615384615385
sk
0.25447316103379725
es
0.21383647798742136
ro
0.21052631578947367
pt
0.19006479481641467
it
0.1758241758241758
fi
0.16666666666666666
reliable
true
passes
true
9
name
"Selvi"
dialogue
"Theek hai."
detectedLang
"vi"
heavyLang
"it"
scores
vi
0.6767241379310345
nl
0.5833333333333334
it
0.5714285714285714
reliable
true
passes
false
10
name
"Hunter"
dialogue
"I have to go. Thanks for your time."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8354059985369422
da
0.734982332155477
tl
0.6922024623803009
no
0.6839887640449438
pl
0.6212121212121212
hr
0.601063829787234
yo
0.5722433460076045
cs
0.5119305856832972
vi
0.4803695150115473
sk
0.4755244755244755
lv
0.45121951219512196
pt
0.43037974683544306
lt
0.4260204081632653
ca
0.38016528925619836
sv
0.38016528925619836
sq
0.3732590529247911
sl
0.34971098265895956
it
0.3382352941176471
nl
0.33234421364985167
ro
0.2857142857142857
ms
0.25742574257425743
be
0.23728813559322032
tr
0.23728813559322032
hu
0.22945205479452058
az
0.21875
et
0.21875
is
0.21875
uk
0.21875
reliable
true
passes
true