Run: 2026-09-03-01M1KCDBBGPRHV8K4096Z73NB6

Run Details

Model:
DeepSeek V4 Pro 0813 (Reasoning, High)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
2m 22s
Cost:
$0.027027
Provider:
Novita

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."

Model response

AI
Marie: "Cette photo... ce n’est pas possible..." Hunter: "Excuse me, miss. You’re Marie, right? I’m Detective Hunter. I need to ask you about last night." Marie: "Quoi ? Non, non... je ne peux pas en parler." Hunter: "I’m sorry—I don’t speak French. Do you understand English at all?" Marie: "Le verger... oui. Mais je n’aurais pas dû y aller. La brume... les cris..." Hunter: "Brume. Fog. Other witnesses mentioned fog. Did you see the men who disappeared?" Marie: "Oui. Deux hommes près de la rivière. Ils sont entrés dans la brume... et ils n’en sont jamais ressortis." Hunter: "I only caught 'disparus.' Disappeared. You have a camera. Did you photograph what happened?" Marie: "Oui. La photo est ici. Mais si je vous la montre, il viendra me chercher." Hunter: "Who is 'he'? Is someone threatening you?" Marie: "L’homme au manteau long. Il m’a vue. Il a dit qu’il reviendrait." Hunter: "A long coat. That matches another report. I need that photo, Marie. Please." Marie: "Non. Pas ici. Quelqu’un nous observe peut-être." Hunter: "Then come with me to the station. I can keep you safe." Marie: "Vous ne comprenez pas. Ce n’est pas un homme ordinaire. La brume l’écoute." Hunter: "I don’t know what that means, but you’re not safe here." *(Hunter’s phone rings.)* Hunter: "Excuse me. Hunter. ... Where? ... I’m on my way." Hunter: "I have to go. Stay in public. Don’t go back to the orchard. I’ll send someone who speaks French." Marie: "Attendez ! Ne me laissez pas seule..." Hunter: "I’ll be back."
created_at"2026-09-03T10:18:11.713Z" 9/3/2026, 10:18:11 AM
model"deepseek/deepseek-v4-pro-20260813"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency1181
moderation_latency(null)
generation_time141931
tokens_prompt303
tokens_completion6856
native_tokens_prompt342
native_tokens_completion6711
native_tokens_completion_images(null)
native_tokens_reasoning6264
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.027027
router(null)
user_agent"langchainjs-openai/1.0.0 ((node/v24.19.0; linux; x64))"
http_referer(null)
request_id"req-1788430691-YMqXMzGjS5rwBIoSagjx"
session_id(null)
preset_id(null)
api_type"completions"
id"gen-1788430691-iosjOrxTawzEfg3JzikI"
upstream_id"d3c8d2c776f51cad8437e1fb732c2a98"
provider_responses
0
endpoint_id"6811c06b-8624-4796-ad6e-807e33b97ae6"
is_byokfalse
latency148
model_permaslug"deepseek/deepseek-v4-pro-20260813"
provider_name"Sail Research"
status429
1
endpoint_id"46f2098d-3994-4c59-b3f9-cbbbf44a2f45"
id"d3c8d2c776f51cad8437e1fb732c2a98"
is_byokfalse
latency410
model_permaslug"deepseek/deepseek-v4-pro-20260813"
provider_name"Novita"
status200
total_cost0.027027
cache_discount(null)
upstream_inference_cost0
provider_name"Novita"
response_cache_source_id(null)
data_region"global"
workspace_id"97e315e5-d303-487d-83c1-83180e8a13d4"

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Marie"
dialogue"Cette photo... ce n’est pas possible..."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8264984227129337
en0.6372031662269129
ca0.6343085106382979
it0.5175438596491229
tl0.4670542635658915
nl0.3861607142857143
ro0.33894230769230765
no0.3159203980099502
sq0.31077694235588976
hu0.30203045685279184
pt0.2930591259640103
da0.2819843342036553
es0.2744063324538259
lt0.2486338797814208
cs0.24242424242424243
de0.24242424242424243
ms0.233983286908078
tr0.22969187675070027
hi0.21875
reliabletrue
passestrue
1
name"Hunter"
dialogue"Excuse me, miss. You’re Marie, right? I’m Detective Hunter. I need to ask you about last night."
detectedLang"en"
heavyLang"en"
scores
en0.8026892721426484
tl0.6584008097165991
no0.5168217609162491
da0.4807692307692307
it0.46808510638297873
ro0.46215139442231074
fr0.46
sq0.424061433447099
ca0.39839572192513373
cs0.39732142857142855
et0.39298561151079137
sv0.3818681318681319
sl0.37615526802218113
hr0.3703358208955224
pt0.3679775280898876
nl0.35344827586206895
yo0.3484555984555985
es0.3395303326810176
pl0.3270189431704885
de0.31402439024390244
sk0.2834394904458599
lv0.26550598476605003
lt0.25742574257425743
ms0.23382519863791143
fi0.16563658838071693
tr0.16044776119402984
be0.09395973154362416
hu0.09029649595687331
az0.08536585365853659
eu0.08536585365853659
is0.08536585365853659
uk0.08536585365853659
vi0.08536585365853659
reliabletrue
passestrue
2
name"Marie"
dialogue"Quoi ? Non, non... je ne peux pas en parler."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8149671052631579
sl0.6611445783132531
nl0.6580547112462006
ca0.6347402597402597
tr0.6299342105263158
hr0.6066433566433567
it0.5856353591160222
sq0.5689655172413793
cs0.5647969052224371
lt0.5408163265306122
es0.49324324324324326
sk0.4886363636363637
da0.4791666666666667
sv0.4791666666666667
no0.46808510638297873
fi0.43324937027707805
ro0.43324937027707805
yo0.4201030927835051
eu0.4186046511627907
lv0.4186046511627907
pl0.412532637075718
en0.4094488188976378
hu0.39839572192513373
az0.3967828418230563
ms0.3951612903225807
de0.3885869565217392
is0.3732590529247911
et0.21875
pt0.21875
reliabletrue
passestrue
3
name"Hunter"
dialogue"I’m sorry—I don’t speak French. Do you understand English at all?"
detectedLang"en"
heavyLang"en"
scores
en0.8090874379534174
tl0.6924969249692496
no0.6031746031746031
da0.5787700084245998
sv0.49647532729103727
cs0.4444444444444444
it0.4212962962962963
pt0.4110718492343934
pl0.40968122786304606
de0.4026284348864994
hr0.3983152827918171
sq0.3924665856622114
sk0.36948297604035313
ro0.3206521739130435
nl0.2625368731563422
vi0.258160237388724
yo0.24812030075187969
et0.23195084485407066
ca0.2163009404388715
is0.21507064364207218
sl0.1935483870967742
es0.1638795986622074
ms0.12587412587412586
be0.12280701754385964
tr0.12280701754385964
hu0.11816578483245149
az0.11190053285968028
lt0.11190053285968028
lv0.11190053285968028
uk0.11190053285968028
reliabletrue
passestrue
4
name"Marie"
dialogue"Le verger... oui. Mais je n’aurais pas dû y aller. La brume... les cris..."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7801943544655252
nl0.5573159366262814
ca0.5564892623716153
es0.5445829338446788
de0.5118191161356629
it0.496288441145281
no0.4952178533475027
yo0.4424882629107981
da0.39720812182741116
sl0.3790849673202614
pt0.3649732620320856
ro0.3649732620320856
tl0.33751743375174337
en0.32045779685264664
sq0.2983751846381093
sv0.2973372781065089
pl0.29210134128166915
vi0.2770167427701674
tr0.2658423493044822
hr0.2647058823529412
lt0.26127527216174184
cs0.23633440514469453
sk0.23387096774193547
hu0.17102966841186737
et0.15026833631484796
ms0.15026833631484796
lv0.1394927536231884
az0.13793103448275862
fi0.12844036697247707
reliabletrue
passestrue
5
name"Hunter"
dialogue"Brume. Fog. Other witnesses mentioned fog. Did you see the men who disappeared?"
detectedLang"en"
heavyLang"en"
scores
en0.7857142857142857
fr0.48478900883218845
tl0.48173741362290223
da0.4620901639344262
nl0.3986254295532647
it0.37796208530805686
sv0.375
no0.36286407766990286
ms0.34375
ca0.3101182654402102
hu0.27083333333333337
et0.2446043165467626
de0.2369186046511628
pt0.2335766423357664
fi0.22680412371134018
ro0.22451994091580502
eu0.20933734939759033
es0.2009132420091324
yo0.16930379746835442
cs0.12935323383084577
sq0.12935323383084577
hr0.11016949152542374
reliabletrue
passestrue
6
name"Marie"
dialogue"Oui. Deux hommes près de la rivière. Ils sont entrés dans la brume... et ils n’en sont jamais ressortis."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8215756244853143
ca0.616519174041298
es0.4546979865771812
da0.44396920444824634
pt0.41016333938294003
it0.40092165898617516
no0.38271604938271603
ro0.32150313152400833
nl0.28017718715393136
et0.2785793562708102
tl0.2753623188405797
ms0.27048260381593714
en0.2571428571428571
lt0.22248803827751198
yo0.22248803827751198
de0.1792929292929293
sq0.17825537294563845
sv0.14921465968586387
tr0.13101604278074866
hu0.12985274431057564
lv0.12516823687752357
vi0.11444141689373298
fi0.10714285714285714
eu0.09722222222222222
az0.08835904628330996
reliabletrue
passestrue
7
name"Hunter"
dialogue"I only caught 'disparus.' Disappeared. You have a camera. Did you photograph what happened?"
detectedLang"en"
heavyLang"en"
scores
en0.7910863509749304
it0.48320413436692505
tl0.441340782122905
fr0.44029850746268656
da0.41972920696324956
ro0.39209726443769
ca0.38524590163934425
sv0.3743482794577685
ms0.36507936507936506
cs0.3442622950819672
no0.3265993265993266
hr0.29494712103407755
nl0.29494712103407755
lt0.28994082840236685
es0.2779783393501805
pl0.2779783393501805
hu0.2709599027946537
de0.26380368098159507
eu0.2583436341161928
sq0.2555831265508685
yo0.2555831265508685
sk0.2518703241895262
tr0.19678714859437751
pt0.1891891891891892
is0.18699186991869918
az0.18144611186903137
lv0.18144611186903137
vi0.18144611186903137
et0.17355371900826447
sl0.13043478260869565
hi0.1137370753323486
be0.1044776119402985
fi0.09502262443438914
uk0.09502262443438914
reliabletrue
passestrue
8
name"Marie"
dialogue"Oui. La photo est ici. Mais si je vous la montre, il viendra me chercher."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8265790434465133
it0.665492957746479
ro0.6100164203612479
es0.5796460176991151
ca0.5401742497579864
tl0.5158002038735984
sq0.5113168724279835
pt0.4814410480349345
yo0.4791666666666667
en0.46509009009009006
no0.4489559164733179
sl0.4391971664698937
de0.433849821215733
cs0.42702050663449936
sk0.4077306733167082
nl0.3725231175693527
sv0.3581081081081081
hr0.326241134751773
lv0.30453879941434847
lt0.3014705882352941
tr0.2313915857605178
fi0.22003284072249588
da0.21617161716171615
az0.19491525423728812
et0.16666666666666666
pl0.1607773851590106
ms0.15026833631484796
vi0.15026833631484796
hi0.1394927536231884
is0.13003663003663005
reliabletrue
passestrue
9
name"Hunter"
dialogue"Who is 'he'? Is someone threatening you?"
detectedLang"en"
heavyLang"en"
scores
en0.8052959501557633
nl0.5894909688013136
tl0.5247148288973384
da0.4692144373673036
is0.4469026548672566
no0.43946188340807174
hu0.3811881188118812
cs0.37810945273631835
sv0.35233160621761656
yo0.32432432432432434
lt0.2492492492492493
es0.2378048780487805
pt0.2378048780487805
et0.22839506172839508
sk0.22839506172839508
reliabletrue
passestrue
10
name"Marie"
dialogue"L’homme au manteau long. Il m’a vue. Il a dit qu’il reviendrait."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7983014861995754
ca0.6431254695717505
ro0.5573159366262814
it0.5539906103286385
es0.4692737430167597
tr0.41430332922318125
en0.40176322418136023
pt0.3964421855146125
az0.375
nl0.3649732620320856
da0.3606998654104979
et0.31358381502890176
hr0.2759146341463415
vi0.26242236024844723
no0.2554858934169279
sl0.250788643533123
hu0.24722662440570525
lv0.2448330683624801
sq0.23878205128205127
tl0.23878205128205127
sv0.2263843648208469
cs0.21746293245469525
sk0.21487603305785125
yo0.20033670033670034
pl0.1810344827586207
lt0.14721723518850988
is0.13636363636363635
eu0.12844036697247707
de0.12361623616236161
fi0.1171003717472119
ms0.1171003717472119
reliabletrue
passestrue
11
name"Hunter"
dialogue"A long coat. That matches another report. I need that photo, Marie. Please."
detectedLang"en"
heavyLang"en"
scores
en0.7974797479747975
tl0.6691176470588235
fr0.6651785714285715
ro0.5909090909090909
sv0.569377990430622
it0.5495495495495496
cs0.533678756476684
da0.5233050847457626
es0.503858875413451
no0.4845360824742268
ca0.48216340621403914
et0.46236559139784944
hr0.46107784431137727
pl0.44785276073619634
sq0.42159383033419023
sk0.4186046511627907
de0.4007989347536618
hu0.39189189189189183
pt0.3869209809264305
yo0.37413073713490963
vi0.3401759530791789
sl0.3293591654247392
lt0.3202416918429003
nl0.3108728943338438
lv0.29906542056074764
ms0.29577464788732394
tr0.24623115577889448
is0.23469387755102045
az0.22813036020583194
eu0.22813036020583194
fi0.21328671328671328
hi0.14611005692599618
be0.1346153846153846
uk0.12280701754385964
reliabletrue
passestrue
12
name"Marie"
dialogue"Non. Pas ici. Quelqu’un nous observe peut-être."
detectedLang"fr"
heavyLang"fr"
scores
fr0.805505685218432
it0.5854591836734694
ca0.5370370370370371
en0.4444444444444444
sq0.27616926503340755
lt0.21875
nl0.21875
eu0.2111650485436893
ms0.20537897310513448
es0.19753086419753088
pt0.19753086419753088
de0.1935483870967742
tr0.1935483870967742
et0.19154228855721395
no0.1792929292929293
pl0.1792929292929293
ro0.1792929292929293
sl0.1792929292929293
reliabletrue
passestrue
13
name"Hunter"
dialogue"Then come with me to the station. I can keep you safe."
detectedLang"en"
heavyLang"en"
scores
en0.8372558838257386
tl0.6531483457844184
no0.6475054229934923
da0.609375
yo0.5997536945812808
it0.5859872611464968
sq0.5678191489361702
sv0.5498614958448753
fr0.5396600566572238
cs0.5370370370370371
pl0.4992295839753467
hr0.49455676516329705
nl0.4921875
ms0.4791666666666667
sl0.45561139028475717
sk0.4472789115646259
lv0.4328097731239092
lt0.4268077601410935
et0.4003690036900369
ca0.3981481481481482
tr0.3879472693032015
az0.38446969696969696
de0.367704280155642
ro0.3394308943089431
fi0.3257261410788381
es0.2388758782201405
pt0.21307506053268765
zh0.19154228855721395
be0.17721518987341772
hu0.1709183673469388
is0.16237113402061853
uk0.16237113402061853
vi0.16237113402061853
reliabletrue
passestrue
14
name"Marie"
dialogue"Vous ne comprenez pas. Ce n’est pas un homme ordinaire. La brume l’écoute."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8108015135878914
ca0.6496815286624203
it0.623545516769336
ro0.5756172839506173
es0.492151431209603
nl0.3949394939493949
tl0.3813273340832396
sq0.3648960739030023
lt0.3436754176610978
en0.3365500603136309
lv0.32926829268292684
yo0.2647058823529412
no0.2434662998624484
et0.23717059639389734
fi0.23184357541899442
eu0.2040520984081042
sl0.1887905604719764
pt0.18154761904761904
hr0.17043740573152338
da0.16666666666666666
tr0.14596273291925466
ms0.13249211356466878
vi0.13249211356466878
cs0.12974683544303797
hu0.12974683544303797
reliabletrue
passestrue
15
name"Hunter"
dialogue"I don’t know what that means, but you’re not safe here."
detectedLang"en"
heavyLang"en"
scores
en0.8389175257731959
tl0.5624270711785297
sq0.3842364532019704
tr0.2829827915869981
no0.27884615384615385
da0.2704280155642024
hr0.2704280155642024
ca0.2690058479532163
sv0.2690058479532163
pl0.251497005988024
it0.23469387755102045
cs0.21875
ro0.1935483870967742
fr0.19006479481641467
yo0.1883116883116883
sk0.1610738255033557
be0.15730337078651685
hu0.1515837104072398
az0.14383561643835618
et0.14383561643835618
is0.14383561643835618
lt0.14383561643835618
lv0.14383561643835618
uk0.14383561643835618
vi0.14383561643835618
reliabletrue
passestrue
16
name"Hunter"
dialogue"Excuse me. Hunter. ... Where? ... I’m on my way."
detectedLang"en"
heavyLang"en"
scores
en0.7942386831275721
tl0.5873452544704264
sq0.48717948717948717
fr0.4755244755244755
et0.453551912568306
fi0.42857142857142855
pl0.3697478991596639
no0.3603411513859275
de0.3562231759656652
cs0.3492407809110629
sk0.3492407809110629
ca0.34497816593886466
pt0.3348115299334811
nl0.3288590604026846
sl0.3288590604026846
it0.3227990970654628
ms0.3227990970654628
es0.2537313432835821
hr0.22680412371134018
hi0.21875
da0.21465968586387435
yo0.2
sv0.1891891891891892
ro0.16434540389972147
tr0.16434540389972147
reliabletrue
passestrue
17
name"Hunter"
dialogue"I have to go. Stay in public. Don’t go back to the orchard. I’ll send someone who speaks French."
detectedLang"en"
heavyLang"en"
scores
en0.8203054806828391
tl0.6737357259380098
da0.46949602122015915
pl0.4657168299198575
yo0.45255474452554745
no0.45155393053016457
ro0.39879759519038077
sv0.3939393939393939
nl0.39086294416243655
it0.3610223642172524
fr0.3541442411194833
hr0.3464052287581699
ca0.3311036789297659
cs0.329608938547486
sq0.3197278911564626
de0.318955732122588
sl0.3142857142857143
et0.3063583815028902
sk0.3047508690614137
lv0.29577464788732394
es0.2907801418439716
pt0.286563614744352
lt0.27710843373493976
ms0.20318725099601595
az0.19678714859437751
eu0.1111111111111111
be0.1044776119402985
tr0.1044776119402985
hu0.10044977511244377
is0.09502262443438914
uk0.09502262443438914
vi0.09502262443438914
reliabletrue
passestrue
18
name"Marie"
dialogue"Attendez ! Ne me laissez pas seule..."
detectedLang"fr"
heavyLang"fr"
scores
fr0.812874251497006
sq0.6069182389937107
it0.5833333333333334
lt0.5667244367417678
ca0.5575221238938053
es0.5201535508637236
en0.5192307692307693
nl0.5029821073558649
no0.4813278008298755
sl0.46351931330472107
lv0.4172494172494172
pt0.40898345153664306
ms0.40191387559808617
fi0.3946731234866829
et0.3902439024390244
tr0.38875305623471884
hr0.3112947658402204
ro0.28977272727272724
de0.2647058823529412
cs0.24698795180722888
da0.24698795180722888
hu0.24698795180722888
tl0.23547400611620795
sv0.20634920634920634
reliabletrue
passestrue
19
name"Hunter"
dialogue"I’ll be back."
detectedLang"en"
heavyLang"en"
scores
en0.8333333333333334
tl0.6666666666666666
hu0.5391705069124424
lt0.5283018867924528
yo0.5238095238095238
reliabletrue
passestrue