Run: 2024-08-09-01J4W2DF3EMBGSAKA35PZ8V3KP

Run Details

Model:
WizardLM 2 8x22b
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
5.6s
Cost:
$0.000376
Provider:
Novita

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."

Model response

AI
Marie: "Je ne comprends pas, je suis désolée." (I don't understand, I'm sorry.) Hunter: "I'm looking for anyone who might have seen something unusual in the past few days." Marie: "Je ne parle pas anglais." (I don't speak English.) Hunter: "Oh, right. You're French." Marie: "Oui." (Yes.) Hunter: "I'm afraid I don't speak French. I'm trying to find out what happened to some missing people." Marie: "Ah, oui. Les disparitions mystérieuses. C'est terrible." (Ah, yes. The mysterious disappearances. It's terrible.) Hunter: "I need to know if you've seen anything suspicious." Marie: "Non, je suis désolée." (No, I'm sorry.) Hunter: "Alright. If you come across any information, call me." [Handing Marie his card] [Hunter's phone rings] Hunter: "I have to take this. Good day." [Walking away] Marie: "Au revoir." (Goodbye.)
id"gen-WhPGMv1xqsswTFfSyJMWTeE1TXpB"
total_cost0.00037568
upstream_id"cmpl-f8165834dd904e57b5027f1a0ec4ca45"
created_at"2024-08-09T17:19:14.277322+00:00" 8/9/2024, 5:19:14 PM
model"microsoft/wizardlm-2-8x22b"
app_id182717
streamedfalse
cancelledfalse
provider_name"Novita"
latency5371
moderation_latency(null)
generation_time0
finish_reason"stop"
tokens_prompt267
tokens_completion232
native_tokens_prompt317
native_tokens_completion270
num_media_prompt(null)
num_media_completion(null)
origin"https://ncbench.com/"
usage0.00037568

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Marie"
dialogue"Je ne comprends pas, je suis désolée."
detectedLang"fr"
heavyLang"fr"
scores
fr0.81089258698941
ca0.5934959349593496
nl0.5519713261648745
sl0.5462794918330308
it0.5395948434622468
lt0.5353159851301116
hr0.5318352059925093
sq0.4877049180327869
cs0.4780793319415449
en0.4658119658119659
ro0.46120689655172414
es0.4529540481400437
tl0.40191387559808617
yo0.3946731234866829
de0.3857493857493857
sk0.3670886075949367
pt0.32795698924731176
lv0.2917847025495751
tr0.2732558139534884
pl0.2668621700879765
fi0.26035502958579887
ms0.251497005988024
hu0.24698795180722888
reliabletrue
passestrue
1
name"Hunter"
dialogue"I'm looking for anyone who might have seen something unusual in the past few days."
detectedLang"en"
heavyLang"en"
scores
en0.8284313725490196
tl0.5783132530120482
da0.4659206510681587
nl0.4570837642192347
no0.44029850746268656
pt0.37275985663082434
ms0.3697478991596639
de0.3644067796610169
yo0.33879093198992444
sv0.32778489116517284
fr0.2981283422459893
es0.22794117647058826
sl0.22680412371134018
it0.21875
ro0.2128935532233883
et0.16666666666666666
az0.13793103448275862
pl0.13793103448275862
sq0.12935323383084577
is0.1191275167785235
reliabletrue
passestrue
2
name"Marie"
dialogue"Je ne parle pas anglais."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8006833712984055
sl0.7485632183908046
lt0.7073578595317725
sq0.6863799283154122
cs0.6800731261425961
hr0.6647509578544061
ca0.642126789366053
sk0.6203904555314533
nl0.6153846153846154
lv0.5939675174013921
it0.5813397129186603
ro0.5813397129186603
yo0.48224852071005914
pl0.47447447447447455
tr0.4664634146341463
hu0.45987654320987653
ms0.45652173913043476
es0.45482866043613707
de0.4496855345911949
en0.4336569579288026
fi0.3346007604562738
az0.2857142857142857
et0.2647058823529412
pt0.2647058823529412
da0.25213675213675213
no0.25213675213675213
sv0.25213675213675213
reliabletrue
passestrue
3
name"Hunter"
dialogue"Oh, right. You're French."
detectedLang"en"
heavyLang"en"
scores
en0.7921615201900237
tl0.6323529411764706
nl0.41666666666666663
es0.358974358974359
pt0.34701492537313433
it0.3055555555555556
da0.29718875502008035
de0.29718875502008035
fr0.29718875502008035
no0.29718875502008035
yo0.29718875502008035
reliabletrue
passestrue
4
name"Marie"
dialogue"Oui."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8076923076923077
reliablefalse
passestrue
5
name"Hunter"
dialogue"I'm afraid I don't speak French. I'm trying to find out what happened to some missing people."
detectedLang"en"
heavyLang"en"
scores
en0.8172959805115714
tl0.5901639344262295
da0.44547134935304994
no0.35965848452508004
pl0.30313588850174217
it0.2998833138856476
hr0.2982456140350877
cs0.2822966507177033
sv0.2805755395683453
yo0.27184466019417475
is0.26380368098159507
ca0.25925925925925924
sk0.2537313432835821
lv0.23566878980891717
nl0.22680412371134018
lt0.21773142112125163
pt0.20844327176781002
sq0.18256130790190736
az0.1758241758241758
et0.1758241758241758
sl0.1678224687933426
ms0.15492957746478872
es0.14040114613180515
ro0.13043478260869565
fi0.11504424778761062
de0.10714285714285714
fr0.10714285714285714
be0.1044776119402985
tr0.1044776119402985
hu0.10044977511244377
uk0.09502262443438914
vi0.09502262443438914
reliabletrue
passestrue
6
name"Marie"
dialogue"Ah, oui. Les disparitions mystérieuses. C'est terrible."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7811816192560175
en0.6396396396396397
ca0.6186844613918017
es0.48186528497409326
tl0.3902439024390244
pt0.38176197836166925
de0.3579454253611557
et0.3127147766323024
it0.30313588850174217
hu0.2831541218637993
no0.2805755395683453
cs0.2647058823529412
sk0.2647058823529412
sq0.25788497217068646
da0.25650557620817843
sv0.25650557620817843
ro0.1935483870967742
lv0.17695473251028807
yo0.17355371900826447
nl0.14346895074946467
az0.13606911447084233
fi0.12854030501089325
pl0.12854030501089325
reliabletrue
passestrue
7
name"Hunter"
dialogue"I need to know if you've seen anything suspicious."
detectedLang"en"
heavyLang"en"
scores
en0.8022598870056498
tl0.5575221238938053
pl0.42716857610474634
hr0.42148760330578516
no0.42052980132450335
da0.40978077571669475
sv0.40476190476190477
cs0.40273037542662116
sq0.4006849315068493
yo0.3902439024390244
ca0.375
sk0.36823104693140796
et0.36594202898550726
lv0.34579439252336447
ro0.3371212121212121
lt0.3230174081237911
hu0.3069306930693069
ms0.3
sl0.25690021231422505
it0.24731182795698922
pt0.2009132420091324
es0.17061611374407584
fr0.17061611374407584
be0.16666666666666666
tr0.16666666666666666
az0.15254237288135594
is0.15254237288135594
uk0.15254237288135594
vi0.15254237288135594
reliabletrue
passestrue
8
name"Marie"
dialogue"Non, je suis désolée."
detectedLang"fr"
heavyLang"fr"
scores
fr0.80719794344473
sl0.5356037151702787
hr0.5327102803738317
yo0.5207667731629393
it0.5033112582781457
cs0.494949494949495
sk0.4915254237288135
nl0.45652173913043476
en0.38271604938271603
pl0.3775933609958506
eu0.3670886075949367
de0.34782608695652173
reliabletrue
passestrue
9
name"Hunter"
dialogue"Alright. If you come across any information, call me."
detectedLang"en"
heavyLang"en"
scores
en0.814716781365802
tl0.630801687763713
fr0.6220302375809936
no0.56575682382134
sv0.5580808080808082
de0.5569620253164557
da0.5489690721649484
it0.5218579234972678
nl0.5172413793103449
sq0.49856733524355296
ca0.4444444444444444
es0.4233937397034596
pt0.41666666666666663
yo0.4117647058823529
cs0.38917975567190227
sl0.38380281690140844
et0.38271604938271603
fi0.30966469428007887
pl0.30966469428007887
lt0.29577464788732394
ms0.29577464788732394
ro0.29149797570850206
lv0.2886178861788618
hr0.2857142857142857
sk0.2842535787321063
eu0.2768595041322314
az0.2647058823529412
zh0.18032786885245902
tr0.16666666666666666
hu0.15254237288135594
reliabletrue
passestrue
10
name"Hunter"
dialogue"I have to take this. Good day."
detectedLang"en"
heavyLang"en"
scores
en0.8479582971329279
tl0.7596153846153846
da0.6902654867256637
pl0.5986238532110092
hr0.5930232558139535
no0.5920745920745921
cs0.5742092457420924
yo0.5614035087719298
sk0.5382585751978892
sl0.5320855614973261
lv0.5138888888888888
lt0.4883040935672514
ca0.44089456869009586
sv0.44089456869009586
sq0.4336569579288026
it0.396551724137931
ro0.33962264150943394
be0.2857142857142857
tr0.2857142857142857
hu0.2768595041322314
az0.2647058823529412
et0.2647058823529412
is0.2647058823529412
uk0.2647058823529412
vi0.2647058823529412
reliabletrue
passestrue
11
name"Marie"
dialogue"Au revoir."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8166259168704157
ro0.7699386503067485
ca0.46428571428571425
cs0.46428571428571425
da0.46428571428571425
en0.46428571428571425
es0.46428571428571425
hr0.46428571428571425
nl0.46428571428571425
no0.46428571428571425
pt0.46428571428571425
sk0.46428571428571425
sl0.46428571428571425
sq0.46428571428571425
reliabletrue
passestrue