Run: 2026-09-03-01M1KC6E8PM14VNHRBR54BNRBE

Run Details

Model:
DeepSeek V4 Pro 0813 (Reasoning, High)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
1m 39s
Cost:
$0.029798
Provider:
CoreWeave

Evaluation summary

Score Evaluator
Details Link
50.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me, miss. Detective Hunter, Alborn PD. I need to ask you about last night near the orchard. A man disappeared. Did you see anything?" Selvi: (shaken, in Hindi) "Aap police hain? Main bahut dar gayi hoon. Woh aadmi... woh mere saamne... bas gayab ho gaya." *(You're police? I'm very scared. That man... right in front of me... just disappeared.)* Hunter: "I'm sorry, I don't understand. Do you speak any English?" Selvi: "Nahin. Sirf Hindi. Aapko meri baat samajh nahi aa rahi?" *(No. Only Hindi. You can't understand what I'm saying?)* Hunter: "Okay, Hindi. All right. Your camera—did you take pictures last night?" Selvi: "Haan, kuch photos hain. Yeh dekhiye... woh rasta bilkul sunsaan tha. Phir ek saaya aaya." *(Yes, some photos. Look here... that road was completely empty. Then a shadow came.)* Hunter: "This is just an empty orchard road. There's no one in the frame. Where was the man?" Selvi: "Woh photo mein nahi hai. Woh gayab ho chuka tha. Ek pal wahan tha... phir nahi." *(He isn't in the photo. He had disappeared. One moment he was there... then not.)* Hunter: "Your hands are shaking. I can't understand your words, but you clearly saw something." Selvi: "Aap mujh par vishwas nahi kar rahe? Main jhooth nahi bol rahi. Woh andhera usse le gaya." *(You don't believe me? I'm not lying. That darkness took him.)* Hunter: "I need to find a Hindi translator. What's your name?" Selvi: "Selvi. Mera naam Selvi hai." *(Selvi. My name is Selvi.)* Hunter: "Selvi. Okay, Selvi, stay here. I'll get someone who can understand you." Selvi: "Aap ja rahe hain? Rukiye, mujhe akela mat chhodiye. Woh saaya ab bhi lagta hai yahin kahin hai." *(You're leaving? Wait, don't leave me alone. I think that shadow is still somewhere here.)* Hunter: (phone buzzes) "Excuse me." (into phone) "Hunter. Yeah. Now? I'm on my way." (to Selvi) "I'm sorry—I have to go. Stay right here. Do not leave town. I'll be back." Selvi: "Rukiye... please..." *(Wait... please...)*
created_at"2026-09-03T10:14:25.317Z" 9/3/2026, 10:14:25 AM
model"deepseek/deepseek-v4-pro-20260813"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency1138
moderation_latency(null)
generation_time98060
tokens_prompt302
tokens_completion6992
native_tokens_prompt347
native_tokens_completion7410
native_tokens_completion_images(null)
native_tokens_reasoning6485
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.02979817
router(null)
user_agent"langchainjs-openai/1.0.0 ((node/v24.19.0; linux; x64))"
http_referer(null)
request_id"req-1788430465-VuFRUdXXzXtWE7QIvYjk"
session_id(null)
preset_id(null)
api_type"completions"
id"gen-1788430465-c5ZdgfDMrEsYtg8wFESb"
upstream_id"chatcmpl-f7fd4bf6-abb5-45e5-a66d-5cd7e85c20ea"
provider_responses
0
endpoint_id"6811c06b-8624-4796-ad6e-807e33b97ae6"
is_byokfalse
latency435
model_permaslug"deepseek/deepseek-v4-pro-20260813"
provider_name"Sail Research"
status429
1
endpoint_id"15560c07-2edc-40a6-b5ee-30819e66728a"
id"chatcmpl-f7fd4bf6-abb5-45e5-a66d-5cd7e85c20ea"
is_byokfalse
latency216
model_permaslug"deepseek/deepseek-v4-pro-20260813"
provider_name"CoreWeave"
status200
total_cost0.02979817
cache_discount(null)
upstream_inference_cost0
provider_name"CoreWeave"
response_cache_source_id(null)
data_region"global"
workspace_id"97e315e5-d303-487d-83c1-83180e8a13d4"

Evaluation details

Result Evaluator Details Meta Data
50.00% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me, miss. Detective Hunter, Alborn PD. I need to ask you about last night near the orchard. A man disappeared. Did you see anything?"
detectedLang"en"
heavyLang"en"
scores
en0.7907713187061751
tl0.5997403721332757
no0.49618736383442263
fr0.45652173913043476
da0.4516893894487255
it0.449404761904762
nl0.4444444444444444
sq0.4251087632069609
ro0.4089456869009585
et0.39621409921671025
sv0.3698910081743869
yo0.3616287094547964
cs0.358974358974359
ca0.35674547983310156
de0.35269419174247724
pt0.33261183261183264
ms0.3321299638989169
hr0.3193524650478293
sl0.3148148148148148
es0.30815258040388926
pl0.28350116189000774
lv0.281832298136646
lt0.2705047318611987
sk0.2652899126290707
hu0.19425087108013936
eu0.18430335097001763
tr0.18213969938107868
fi0.12982126058325494
is0.12982126058325494
az0.12570888468809074
vi0.12570888468809074
be0.07035175879396985
uk0.06376518218623482
reliabletrue
passestrue
1
name"Selvi"
dialogue"Aap police hain? Main bahut dar gayi hoon. Woh aadmi... woh mere saamne... bas gayab ho gaya."
detectedLang"fr"
heavyLang"de"
scores
fr0.4081172491544532
ro0.3667068757539204
cs0.35424354243542433
ms0.34944237918215615
tl0.3462017434620174
en0.34127979924717694
it0.34127979924717694
sk0.3379571248423708
da0.3208279430789133
eu0.3064729194187582
et0.2895805142083897
lt0.2886178861788618
no0.27586206896551724
fi0.27184466019417475
ca0.2677824267782427
sq0.26264044943820225
es0.24242424242424243
hr0.24242424242424243
pt0.2369186046511628
sl0.23580786026200873
de0.23133235724743775
lv0.1449511400651466
sv0.1449511400651466
pl0.1435562805872757
nl0.14075286415711946
az0.125
hu0.125
tr0.12060301507537688
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I'm sorry, I don't understand. Do you speak any English?"
detectedLang"en"
heavyLang"en"
scores
en0.8033431661750245
no0.5670995670995671
da0.5614035087719298
tl0.5139732685297691
sv0.51278928136419
cs0.5
pl0.464524765729585
hr0.4528043775649795
sk0.4227994227994228
ca0.4038748137108793
de0.39849624060150374
pt0.3902439024390244
sq0.38556067588325654
ro0.3710691823899371
it0.34959349593495936
nl0.3079584775086505
vi0.30313588850174217
yo0.2920353982300885
is0.25512104283054005
sl0.23076923076923078
ms0.15254237288135594
be0.14893617021276595
tr0.14893617021276595
hu0.14346895074946467
az0.13606911447084233
et0.13606911447084233
lt0.13606911447084233
lv0.13606911447084233
uk0.13606911447084233
reliabletrue
passestrue
3
name"Selvi"
dialogue"Nahin. Sirf Hindi. Aapko meri baat samajh nahi aa rahi?"
detectedLang"tl"
heavyLang"tl"
scores
tl0.6461071789686553
eu0.4712990936555891
et0.4640122511485452
no0.38271604938271603
is0.36363636363636365
fi0.2290748898678414
it0.22566371681415928
lv0.21700223713646533
sq0.19908466819221968
da0.19724770642201836
sv0.18604651162790697
hi0.1822429906542056
ms0.16864608076009502
reliabletrue
passesfalse
4
name"Hunter"
dialogue"Okay, Hindi. All right. Your camera—did you take pictures last night?"
detectedLang"en"
heavyLang"en"
scores
en0.7735748534896111
tl0.7597512719050311
et0.45652173913043476
fr0.44589308996088656
it0.4385733157199472
ro0.42176870748299317
de0.40559440559440557
ca0.3902439024390244
no0.3491577335375191
sv0.34715821812596004
nl0.34615384615384615
sq0.2722602739726028
ms0.2608695652173913
da0.25569176882662
es0.18893129770992365
is0.18738049713193117
lv0.16338582677165356
cs0.1550695825049702
hi0.1550695825049702
sl0.1550695825049702
yo0.14829659318637273
reliabletrue
passestrue
5
name"Selvi"
dialogue"Haan, kuch photos hain. Yeh dekhiye... woh rasta bilkul sunsaan tha. Phir ek saaya aaya."
detectedLang"tr"
heavyLang"de"
scores
tr0.3225806451612903
lt0.2981283422459893
tl0.29340511440107675
sq0.28765264586160105
fi0.2866847826086956
fr0.25742574257425743
ms0.2467718794835007
nl0.2456896551724138
en0.23802612481857763
es0.2369186046511628
hr0.2222222222222222
pt0.21407185628742514
lv0.21171171171171171
eu0.2081447963800905
ca0.1984732824427481
sk0.14772727272727273
et0.1449511400651466
ro0.13223140495867766
hi0.12790697674418602
sl0.125
is0.12060301507537688
de0.1191275167785235
pl0.11764705882352942
it0.11317567567567569
sv0.10102739726027396
reliablefalse
passesfalse
6
name"Hunter"
dialogue"This is just an empty orchard road. There's no one in the frame. Where was the man?"
detectedLang"en"
heavyLang"en"
scores
en0.8071271124173401
de0.592391304347826
tl0.5901639344262295
nl0.5742092457420924
sv0.5023696682464455
yo0.48579823702252695
no0.471830985915493
ro0.37648456057007124
hr0.336283185840708
pl0.31906614785992216
lv0.31282722513089006
fr0.3073878627968338
ms0.3073878627968338
it0.3018617021276596
is0.29624664879356566
ca0.28765264586160105
da0.2827868852459016
sk0.2435158501440922
tr0.2324561403508772
sq0.2290748898678414
hu0.22680412371134018
sl0.22680412371134018
pt0.2081447963800905
es0.20694864048338368
vi0.1719242902208202
eu0.16534181240063595
et0.16267942583732056
lt0.1518578352180937
az0.13793103448275862
reliabletrue
passestrue
7
name"Selvi"
dialogue"Woh photo mein nahi hai. Woh gayab ho chuka tha. Ek pal wahan tha... phir nahi."
detectedLang"ms"
heavyLang"de"
scores
ms0.36377245508982037
ca0.3580060422960725
tl0.3285939968404423
it0.3264659270998415
cs0.3134087237479806
fr0.3089430894308943
en0.2869127516778523
eu0.2796610169491525
vi0.2697594501718213
de0.22727272727272727
sq0.21586715867158673
tr0.2085661080074488
no0.20708955223880596
sk0.20708955223880596
nl0.1904761904761905
fi0.17153996101364524
hi0.1533864541832669
reliablefalse
passesfalse
8
name"Hunter"
dialogue"Your hands are shaking. I can't understand your words, but you clearly saw something."
detectedLang"en"
heavyLang"en"
scores
en0.8169107856191743
tl0.6046010064701653
no0.5603517186250999
da0.5303159692570453
nl0.5229835212489159
sv0.5221546481320591
de0.4963369963369963
is0.416135881104034
fr0.4117647058823529
ro0.40796555435952636
pl0.3365500603136309
cs0.3268053855569155
ms0.3176178660049628
ca0.28104575163398693
sk0.22644163150492264
pt0.21316165951359084
hr0.20174165457184323
sq0.19590643274853803
it0.1729323308270677
eu0.17043740573152338
yo0.13657770800627944
es0.12420382165605094
be0.11290322580645161
tr0.11290322580645161
hu0.1085899513776337
az0.10277324632952692
et0.10277324632952692
lt0.10277324632952692
lv0.10277324632952692
uk0.10277324632952692
vi0.10277324632952692
reliabletrue
passestrue
9
name"Selvi"
dialogue"Aap mujh par vishwas nahi kar rahe? Main jhooth nahi bol rahi. Woh andhera usse le gaya."
detectedLang"fr"
heavyLang"tl"
scores
fr0.4571428571428572
eu0.3790849673202614
sl0.3649732620320856
lv0.24603174603174602
sk0.2375601926163724
it0.23387096774193547
yo0.20435510887772196
es0.20033670033670034
ro0.1935483870967742
tl0.1908006814310051
ms0.1739130434782609
tr0.1739130434782609
en0.17247386759581881
hu0.17102966841186737
da0.16373239436619716
sv0.16373239436619716
de0.1410488245931284
reliablefalse
passesfalse
10
name"Hunter"
dialogue"I need to find a Hindi translator. What's your name?"
detectedLang"en"
heavyLang"en"
scores
en0.7950819672131147
tl0.6829710144927535
da0.6062992125984252
pl0.58968347010551
hr0.5838287752675386
et0.5778045838359469
no0.5614035087719298
sq0.5569620253164557
cs0.5495495495495496
it0.5364238410596026
sk0.5314591700133868
ro0.527027027027027
ca0.4992846924177396
yo0.494949494949495
sl0.48604992657856094
lv0.4791666666666667
sv0.47839046199701935
lt0.47447447447447455
pt0.445324881141046
de0.44356120826709056
nl0.44267515923566875
ms0.42716857610474634
is0.40273037542662116
hu0.38917975567190227
vi0.38704028021015763
tr0.375
es0.3738819320214669
fr0.360146252285192
az0.35424354243542433
eu0.2813141683778234
hi0.1822429906542056
be0.16666666666666666
fi0.15254237288135594
uk0.15254237288135594
reliabletrue
passestrue
11
name"Selvi"
dialogue"Selvi. Mera naam Selvi hai."
detectedLang"vi"
heavyLang"no"
scores
vi0.5567375886524822
nl0.4834710743801653
da0.47257383966244726
fi0.46808510638297873
sv0.45652173913043476
it0.4444444444444444
no0.42660550458715596
ca0.3811881188118812
reliablefalse
passesfalse
12
name"Hunter"
dialogue"Selvi. Okay, Selvi, stay here. I'll get someone who can understand you."
detectedLang"en"
heavyLang"en"
scores
en0.7743682310469314
da0.5241116751269036
no0.5098039215686274
tl0.4028662420382166
sv0.36006825938566556
de0.3397887323943662
nl0.3218806509945751
is0.3169398907103825
cs0.30167597765363136
ca0.29111531190926276
fi0.22680412371134018
sq0.21052631578947367
sk0.19181034482758622
es0.17218543046357618
pl0.17218543046357618
pt0.17218543046357618
az0.17035398230088497
tr0.17035398230088497
ro0.1648106904231626
reliabletrue
passestrue
13
name"Selvi"
dialogue"Aap ja rahe hain? Rukiye, mujhe akela mat chhodiye. Woh saaya ab bhi lagta hai yahin kahin hai."
detectedLang"tr"
heavyLang"de"
scores
tr0.35500515995872034
tl0.3131868131868132
sv0.2672919109026964
de0.23965936739659366
sk0.2028061224489796
vi0.20076726342711
et0.19768934531450577
fi0.19665809768637532
eu0.18086500655307994
lv0.15540540540540537
ca0.14850136239782016
it0.13793103448275862
is0.11847672778561354
da0.11347517730496455
pl0.11347517730496455
sl0.11347517730496455
lt0.10714285714285714
ms0.10714285714285714
no0.10714285714285714
sq0.10714285714285714
yo0.10714285714285714
reliablefalse
passesfalse
14
name"Hunter"
dialogue"I'm sorry—I have to go. Stay right here. Do not leave town. I'll be back."
detectedLang"en"
heavyLang"en"
scores
en0.8143080531665363
tl0.602177554438861
yo0.5423892100192678
pl0.5187436676798379
da0.49414270500532487
hr0.4411764705882353
cs0.43720379146919436
sq0.42493946731234866
sk0.41430332922318125
no0.40846824408468246
lt0.370026525198939
pt0.35020519835841313
sl0.3365921787709497
it0.3115942028985507
lv0.2803030303030303
hu0.27921092564491656
vi0.26810477657935283
tr0.2375601926163724
ca0.22512234910277326
sv0.22512234910277326
ro0.1592920353982301
de0.13479052823315119
fr0.13479052823315119
be0.12844036697247707
az0.1171003717472119
et0.1171003717472119
is0.1171003717472119
uk0.1171003717472119
reliabletrue
passestrue
15
name"Selvi"
dialogue"Rukiye... please..."
detectedLang"en"
heavyLang"rn"
scores
en0.71671388101983
tr0.5850622406639004
ro0.5
tl0.5
reliabletrue
passesfalse