NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2HDVKHB5MKE9AXGSPZ6YE
Run Details
Model:
Mistral NeMO
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
3.7s
Cost:
$0.000102
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet** | Character | Lines | |---|---| | **Selvi** | (Looking at her camera, troubled) "मैंने कुछ देखा था, लेकिन मैं नहीं जानती कि क्या करना चाहिए।" (I saw something, but I don't know what to do.) | | | (Shaken) "यहां कुछ गलत है, मैं जान सकती हूँ।" (Something is wrong here, I can feel it.) | | **Hunter** | (Approaching her) "Excuse me, miss. I couldn't help but notice you seem upset. Mind if I sit down?" | | | (Confused, after Selvi responds in Hindi) "I'm sorry, I don't understand. Do you speak English?" | | | (Trying to communicate) "I'm Detective Hunter. I'm investigating some recent events in town. Did you see or hear something?" | | **Selvi** | (Nods, then speaks in Hindi) "मैंने एक आदमी को देखा था, जो रात को अकेले चल रहा था। वह बहुत जल्दी में लग रहा था।" (I saw a man walking alone at night. He seemed to be in a hurry.) | | **Hunter** | (Still confused, but trying to understand) "A man? Walking alone at night? Do you know where he was going?" | | | (Phone rings) "I'm sorry, I have to take this. (Answers phone) Yes? (Listens) I'll be right there. (Hangs up) I'm sorry, I have to go. But please, if you remember anything else, find me. I'm staying at the Alborn Inn. (Stands up) Thank you for your time. (Begins to leave) | | **Selvi** | (Watching Hunter leave, worried) "मैं क्या करूँ, मैं नहीं जानती।" (What should I do, I don't know.) |
total_cost
0.00010244999999999998
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Selvi"
dialogue
"मैंने कुछ देखा था, लेकिन मैं नहीं जानती कि क्या करना चाहिए।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8893547635686
mr
0.8824839188520535
reliable
true
passes
true
1
name
"Hunter"
dialogue
"Excuse me, miss. I couldn't help but notice you seem upset. Mind if I sit down?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.758220502901354
fr
0.4731296101159114
tl
0.453551912568306
no
0.40191387559808617
it
0.36224489795918363
sq
0.36061381074168797
da
0.3498049414824447
cs
0.34810951760104303
et
0.3297587131367293
sv
0.32249322493224936
hr
0.3084370677731674
es
0.27641099855282203
ro
0.274310595065312
lv
0.2668621700879765
fi
0.2647058823529412
pt
0.24471299093655588
hu
0.2366412213740458
sk
0.2260061919504644
ca
0.2163009404388715
pl
0.20127795527156547
sl
0.14965986394557826
yo
0.1482112436115843
ms
0.14383561643835618
nl
0.14383561643835618
be
0.12280701754385964
tr
0.12280701754385964
az
0.11190053285968028
is
0.11190053285968028
lt
0.11190053285968028
uk
0.11190053285968028
vi
0.11190053285968028
reliable
true
passes
true
2
name
"Selvi"
dialogue
"मैंने एक आदमी को देखा था, जो रात को अकेले चल रहा था। वह बहुत जल्दी में लग रहा था।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.8821712985535511
mr
0.8770144189991518
reliable
true
passes
true
3
name
"Hunter"
dialogue
"A man? Walking alone at night? Do you know where he was going?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8328380386329867
tl
0.6411483253588517
de
0.5426829268292682
no
0.5150862068965517
yo
0.5054945054945055
da
0.5
pl
0.4972067039106145
nl
0.4742990654205608
pt
0.4492044063647491
cs
0.4325346784363178
fr
0.4296577946768061
sq
0.3723849372384937
sk
0.3679775280898876
sv
0.3670886075949367
hr
0.3401759530791789
ca
0.3353028064992614
es
0.32432432432432434
it
0.3202416918429003
sl
0.3181818181818182
vi
0.2868462757527734
lv
0.2834394904458599
lt
0.281150159744409
hu
0.25742574257425743
fi
0.2549668874172185
is
0.24496644295302009
ro
0.23599320882852293
tr
0.14611005692599618
az
0.1346153846153846
eu
0.1346153846153846
et
0.12280701754385964
ms
0.12280701754385964
reliable
true
passes
true
4
name
"Selvi"
dialogue
"मैं क्या करूँ, मैं नहीं जानती।"
detectedLang
"hi"
heavyLang
"hi"
scores
hi
0.889651790093183
mr
0.8837209302325582
reliable
true
passes
true