Run: 2024-08-09-01J4W2GQX4QHPN6D3MCVVTX1E2

Run Details

Model:
DeepSeek-V2 Chat
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
10.7s
Cost:
$0.000095
Provider:
DeepSeek

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."

Model response

AI
Marie: "Je suis désolée, mais je ne comprends pas l'anglais." Hunter: "Excuse me, miss. Have you seen anything unusual around here lately?" Marie: "Je suis très inquiète. Il y a eu des choses étranges ces derniers jours." Hunter: "I'm sorry, I don't understand. Can you speak English?" Marie: "Non, je ne parle pas anglais. Je suis désolée." Hunter: "Alright, I'll try to find someone who can translate. Thank you." Marie: "De rien. Bonne chance avec votre enquête." Hunter: "Thanks... I guess. I gotta go, I just got a call." Marie: "Au revoir." Hunter: "Goodbye."
id"gen-RvFRuUVI6cvKNirXZAJRTn2nVZYl"
total_cost0.00009492
upstream_id"cc869e3800294b91ba09fdd5f01d542a"
created_at"2024-08-09T17:21:06.746827+00:00" 8/9/2024, 5:21:06 PM
model"deepseek/deepseek-chat"
app_id182717
streamedfalse
cancelledfalse
provider_name"DeepSeek"
latency10576
moderation_latency(null)
generation_time0
finish_reason"stop"
tokens_prompt272
tokens_completion166
native_tokens_prompt292
native_tokens_completion193
num_media_prompt(null)
num_media_completion(null)
origin"https://ncbench.com/"
usage0.00009492

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Marie"
dialogue"Je suis désolée, mais je ne comprends pas l'anglais."
detectedLang"fr"
heavyLang"fr"
scores
fr0.802705749718151
ca0.5721271393643031
it0.521203830369357
lt0.5111731843575419
nl0.46808510638297873
sl0.46236559139784944
pt0.45567651632970446
hr0.4479495268138801
sq0.40476190476190477
cs0.3955094991364421
en0.38380281690140844
ro0.3794326241134752
es0.37163375224416517
lv0.360146252285192
tl0.32432432432432434
yo0.317738791423002
de0.30966469428007887
sk0.29292929292929293
tr0.21171171171171171
pl0.20634920634920634
fi0.2009132420091324
ms0.1935483870967742
hu0.1898148148148148
reliabletrue
passestrue
1
name"Hunter"
dialogue"Excuse me, miss. Have you seen anything unusual around here lately?"
detectedLang"en"
heavyLang"en"
scores
en0.7949831162566329
tl0.48296836982968366
sq0.4501940491591203
ms0.35993975903614456
et0.32
nl0.3122977346278317
es0.2963576158940397
pt0.2796610169491525
fi0.2647058823529412
da0.256993006993007
fr0.17635658914728683
no0.17635658914728683
sl0.17153996101364524
it0.15841584158415842
cs0.1550695825049702
sk0.1550695825049702
de0.15169660678642716
hi0.15169660678642716
pl0.15169660678642716
reliabletrue
passestrue
2
name"Marie"
dialogue"Je suis très inquiète. Il y a eu des choses étranges ces derniers jours."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8199185718759786
en0.5721726190476191
ca0.5251857968620974
de0.47440585009140773
it0.4390243902439025
nl0.42152917505030174
pt0.3960084033613445
hr0.3837084673097535
cs0.37837837837837834
sk0.36882546652030734
sl0.36743674367436746
ro0.3502824858757062
es0.3473325766174802
pl0.343607305936073
da0.3413516609392898
sv0.33449074074074076
yo0.3290548424737456
tr0.3282710280373832
az0.3105515587529976
hu0.27944862155388467
et0.27122940430925224
no0.27122940430925224
vi0.22611036339165547
ms0.2144808743169399
tl0.20249653259361997
fi0.18899858956276447
lt0.1248097412480974
is0.11538461538461538
sq0.11128284389489954
eu0.10852713178294573
lv0.10852713178294573
reliabletrue
passestrue
3
name"Hunter"
dialogue"I'm sorry, I don't understand. Can you speak English?"
detectedLang"en"
heavyLang"en"
scores
en0.8019017432646592
no0.5828698553948832
da0.5772266065388951
tl0.5300751879699248
sv0.5288944723618091
cs0.41588785046728977
de0.4140625
ro0.3862520458265139
ca0.3644067796610169
it0.3644067796610169
hr0.36006825938566556
pl0.35233160621761656
nl0.3218806509945751
sk0.30037313432835816
tr0.2816091954022989
az0.27184466019417475
is0.267578125
sq0.2632612966601179
pt0.21052631578947367
yo0.1883116883116883
ms0.1610738255033557
be0.15730337078651685
hu0.1515837104072398
et0.14383561643835618
lt0.14383561643835618
lv0.14383561643835618
uk0.14383561643835618
vi0.14383561643835618
reliabletrue
passestrue
4
name"Marie"
dialogue"Non, je ne parle pas anglais. Je suis désolée."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8015873015873016
sl0.6345919610231425
lt0.5850622406639004
it0.5683453237410072
sq0.5607613469985359
cs0.5535714285714286
hr0.5363214837712519
ca0.511400651465798
sk0.48805460750853247
nl0.4827586206896552
lv0.460431654676259
ro0.44751381215469616
en0.4307400379506641
yo0.35205183585313177
pl0.34497816593886466
tr0.33774834437086093
hu0.33184855233853006
ms0.3288590604026846
es0.3273542600896861
de0.3227990970654628
fi0.22680412371134018
eu0.2248062015503876
az0.1891891891891892
et0.17355371900826447
pt0.17355371900826447
da0.16434540389972147
no0.16434540389972147
sv0.16434540389972147
reliabletrue
passestrue
5
name"Hunter"
dialogue"Alright, I'll try to find someone who can translate. Thank you."
detectedLang"en"
heavyLang"en"
scores
en0.801980198019802
tl0.5031055900621118
da0.4601889338731444
no0.44521497919556174
cs0.40119760479041916
hr0.38837920489296635
vi0.36808846761453395
yo0.3453355155482815
pl0.33884297520661155
sk0.3277310924369748
lv0.32088285229202035
sl0.3103448275862069
lt0.29947460595446584
ro0.2945326278659612
nl0.2895204262877442
pt0.2831541218637993
es0.27927927927927926
fr0.2779783393501805
ca0.2753623188405797
de0.26062846580406657
tr0.25925925925925924
it0.25650557620817843
az0.2537313432835821
sq0.15254237288135594
eu0.14346895074946467
et0.12854030501089325
ms0.12854030501089325
sv0.12854030501089325
reliabletrue
passestrue
6
name"Marie"
dialogue"De rien. Bonne chance avec votre enquête."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8381877022653722
pt0.5714285714285714
de0.4915254237288135
nl0.4863013698630137
en0.46619217081850534
da0.45848375451263534
it0.4230769230769231
es0.41972920696324956
tl0.41972920696324956
sv0.402390438247012
no0.358974358974359
yo0.33774834437086093
ca0.336283185840708
ro0.33184855233853006
sq0.2890995260663507
tr0.24623115577889448
hu0.24433249370277077
eu0.1891891891891892
cs0.18256130790190736
az0.17355371900826447
ms0.17355371900826447
sl0.17355371900826447
hr0.16434540389972147
sk0.16434540389972147
reliabletrue
passestrue
7
name"Hunter"
dialogue"Thanks... I guess. I gotta go, I just got a call."
detectedLang"en"
heavyLang"en"
scores
en0.7725549658832449
it0.5934959349593496
sv0.5588235294117647
ca0.5391705069124424
pl0.5305164319248826
tl0.5283018867924528
hu0.5245641838351823
yo0.4854202401372213
vi0.4809688581314879
hr0.455535390199637
cs0.441340782122905
ro0.43289224952741023
et0.4318181818181818
no0.4174757281553398
da0.4129158512720157
sq0.40711462450592883
sk0.402390438247012
nl0.3775933609958506
fi0.3684210526315789
tr0.3288590604026846
lt0.3258426966292135
pt0.3181818181818182
is0.3150684931506849
az0.3071593533487298
lv0.3071593533487298
es0.3055555555555556
fr0.2805755395683453
sl0.23076923076923078
be0.1891891891891892
eu0.1891891891891892
de0.18256130790190736
ms0.17355371900826447
uk0.17355371900826447
reliabletrue
passestrue
8
name"Marie"
dialogue"Au revoir."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8166259168704157
ro0.7699386503067485
ca0.46428571428571425
cs0.46428571428571425
da0.46428571428571425
en0.46428571428571425
es0.46428571428571425
hr0.46428571428571425
nl0.46428571428571425
no0.46428571428571425
pt0.46428571428571425
sk0.46428571428571425
sl0.46428571428571425
sq0.46428571428571425
reliabletrue
passestrue
9
name"Hunter"
dialogue"Goodbye."
detectedLang"en"
heavyLang"en"
scores
en0.6774193548387096
reliablefalse
passestrue