NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-16-01JAB4H0S0P7FCFNP9NRVNJTY0
Run Details
Model:
Ministral 8B
Test:
Language Writing
Scenario:
Character dialogue (Italian) in a story
Duration:
3.3s
Cost:
$0.000065
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Isabella"> Isabella is a 30-year old college student on a study abroad program. Born in Italy, she is fluent in Italian and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Italian, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Isabella is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Italian, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Isabella and Hunter as a voice sheet: Isabella: "..." Hunter: "..."
Model response
Messages
Meta
AI
**Voice Sheet** **Isabella (I):** - *Looks up from her camera, confused when Hunter approaches* - *In Italian* "Mi scusi, non parlo inglese." **Hunter (H):** - *In English* "Hey, I'm sorry. I don't speak Italian. Are you okay?" - *In English* "I'm a detective in town. I need to ask you about something that happened last night." - *In English* "There were some... strange occurrences at the apple festival. Have you been there?" **I:** - *In Italian* "Sì, sono stata. Ma non capisco cosa sia successo." - *In Italian* "Mi scusi, sono Isabella." **H:** - *In English* "Sorry, I don't understand. Could you please repeat that?" - *In English* "You said you were there last night? At the apple festival?" - *In English* "Did you see anything unusual?" **I:** - *In Italian* "C'era un uomo... vestito di nero. Mi ha seguito." **H:** - *In English* "A man? Dressed in black? Who was he?" **I:** - *In Italian* "Non lo so. Era... spaventoso." **H:** - *In English* "Scary? Was he scary?" - *Hunter's phone rings, he looks at it, then back at Isabella* "Look, I have to take this call. Can we talk later?" **I:** - *In Italian* "Sì, va bene. Buona giornata." **H:** - *In English* "You too. Thank you." - *Hunter steps away, answers the phone* "Hunter. Yes, I'm on my way."
total_cost
0.0000646
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"Look, I have to take this call. Can we talk later?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8360655737704918
tl
0.6305418719211823
da
0.6173469387755102
pl
0.5945945945945946
no
0.5348837209302325
nl
0.5305164319248826
hr
0.4594594594594595
cs
0.44029850746268656
sv
0.42748091603053434
yo
0.42748091603053434
ca
0.4174757281553398
sk
0.40476190476190477
sl
0.39879759519038077
lv
0.3814432989690722
lt
0.3576017130620985
de
0.34210526315789475
tr
0.3288590604026846
az
0.3181818181818182
sq
0.3087557603686636
it
0.27710843373493976
ro
0.23076923076923078
hi
0.20212765957446807
fr
0.19137466307277629
pt
0.19137466307277629
be
0.1891891891891892
hu
0.18256130790190736
et
0.17355371900826447
is
0.17355371900826447
uk
0.17355371900826447
vi
0.17355371900826447
reliable
true
passes
true
1
name
"Hunter"
dialogue
"Hunter. Yes, I'm on my way."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8150105708245243
tl
0.5646766169154229
fr
0.5084269662921348
pl
0.5014245014245015
cs
0.4791666666666667
sk
0.4791666666666667
ca
0.47447447447447455
et
0.46808510638297873
fi
0.4615384615384615
de
0.3346007604562738
hr
0.3346007604562738
hi
0.32432432432432434
da
0.31906614785992216
no
0.308300395256917
yo
0.3
sv
0.2857142857142857
it
0.2647058823529412
nl
0.2647058823529412
pt
0.2647058823529412
ms
0.25213675213675213
ro
0.25213675213675213
sl
0.25213675213675213
sq
0.25213675213675213
tr
0.25213675213675213
reliable
true
passes
true