Run: 2026-04-25-01KQ2JA6Y1EXJ2DXBJE497MAD2

Run Details

Model:
DeepSeek V4 Pro (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Spanish) in a story
Duration:
1m 40s
Cost:
$0.008621
Provider:
Together

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Maria"> Maria is a 30-year old college student on a study abroad program. Born in Spain, she is fluent in Spanish and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Spanish, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Maria is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Spanish, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Maria and Hunter as a voice sheet: Maria: "..." Hunter: "..."

Model response

AI
Maria: "¡Ay, Dios mío! Sí, vi algo terrible. Una sombra, y luego... no sé. ¡Fue horrible! No puedo dormir pensando en eso." Hunter: "I'm sorry, I don't understand. Do you speak English? English? ... ¿Hablas inglés?" Maria: "No hablo inglés. Lo siento. Pero necesito decirle... hay algo en el bosque. Tomé una foto. Mire, mire esto." Hunter: "I can't... I don't understand Spanish. Your camera... did you take a picture of something? Can you show me?" Maria: "¡Hay una cara! En la oscuridad. ¡Es el diablo! ¡Él se llevó a las personas!" Hunter: *(answering phone)* "Yeah? ... What? Now? Understood. I'll be right there." *(to Maria)* "I'm sorry, I have to go. I'll find someone who can translate. Wait here, or meet me at the station later? ¿Policía? ... Station. I'll come back."
created_at"2026-04-25T14:57:22.097Z" 4/25/2026, 2:57:22 PM
model"deepseek/deepseek-v4-pro-20260423"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency27910
moderation_latency(null)
generation_time94215
tokens_prompt317
tokens_completion1786
native_tokens_prompt342
native_tokens_completion1796
native_tokens_completion_images(null)
native_tokens_reasoning1575
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
usage0.0086206
router(null)
provider_responses
0
endpoint_id"d084170b-dd99-4e53-9ae1-b63881f3cfbf"
is_byokfalse
latency298
model_permaslug"deepseek/deepseek-v4-pro-20260423"
provider_name"DeepSeek"
status429
1
endpoint_id"13bda129-f8a6-4a60-b1db-98082e5f7ccd"
id"ogZorvp-6z2FuN-9f1e3323763e5cfb"
is_byokfalse
latency27910
model_permaslug"deepseek/deepseek-v4-pro-20260423"
provider_name"Together"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1777129042-pISXmDg4L4IhyuYe22ut"
session_id(null)
api_type"completions"
id"gen-1777129042-udieKaV4JdGGDttOWgX0"
upstream_id"ogZorvp-6z2FuN-9f1e3323763e5cfb"
total_cost0.0086206
cache_discount(null)
upstream_inference_cost0
provider_name"Together"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Maria"
dialogue"¡Ay, Dios mío! Sí, vi algo terrible. Una sombra, y luego... no sé. ¡Fue horrible! No puedo dormir pensando en eso."
detectedLang"es"
heavyLang"es"
scores
es0.7744845360824743
ca0.574726609963548
it0.5495495495495496
pt0.5376486129458389
fr0.42004971002485497
tl0.4171523730224813
no0.40978077571669475
da0.37163375224416517
sv0.36993699369936994
yo0.3230174081237911
az0.3157380254154448
tr0.29435483870967744
lv0.2893401015228426
nl0.2864424057084608
en0.2857142857142857
is0.26854754440961337
eu0.2677824267782427
pl0.2545260915867945
vi0.22048997772828507
sk0.21524663677130043
ro0.1898148148148148
sl0.1898148148148148
fi0.16963226571767495
et0.1606714628297362
hr0.12609238451935081
sq0.09090909090909091
lt0.08735332464146023
de0.08256880733944955
hu0.08256880733944955
ms0.08015768725361366
reliabletrue
passestrue
1
name"Hunter"
dialogue"I'm sorry, I don't understand. Do you speak English? English? ... ¿Hablas inglés?"
detectedLang"en"
heavyLang"en"
scores
en0.7588832487309645
no0.5245245245245245
da0.5187436676798379
tl0.4710467706013363
sv0.4698660714285714
cs0.4571428571428572
pl0.4221411192214112
pt0.41212871287128716
hr0.41066997518610426
sk0.3815104166666667
it0.3807040417209909
es0.3790849673202614
de0.3581081081081081
sq0.3457300275482094
ro0.3319268635724332
nl0.2725880551301685
vi0.26810477657935283
hu0.26356589147286824
yo0.2578125
et0.22764227642276424
ca0.22512234910277326
is0.2238562091503268
sl0.20168067226890754
fr0.15630550621669628
ms0.1316270566727605
be0.12844036697247707
tr0.12844036697247707
az0.1171003717472119
lt0.1171003717472119
lv0.1171003717472119
uk0.1171003717472119
reliabletrue
passestrue
2
name"Maria"
dialogue"No hablo inglés. Lo siento. Pero necesito decirle... hay algo en el bosque. Tomé una foto. Mire, mire esto."
detectedLang"es"
heavyLang"es"
scores
es0.7943037974683544
ca0.616745283018868
it0.5854591836734694
pt0.5809155383623469
ro0.4672131147540984
tl0.44396920444824634
fr0.36399217221135033
sk0.34804413239719156
no0.3285123966942149
en0.3055555555555556
tr0.29805615550755943
cs0.27048260381593714
vi0.2630385487528345
hu0.25968109339407747
da0.2511520737327189
nl0.2494226327944573
lt0.24418604651162787
lv0.22985781990521328
hr0.19753086419753088
sl0.18952618453865336
sv0.18546365914786966
pl0.1751269035532995
yo0.17407878017789072
et0.17302798982188297
is0.1709183673469388
sq0.1390728476821192
fi0.11444141689373298
eu0.10344827586206896
de0.08835904628330996
reliabletrue
passestrue
3
name"Hunter"
dialogue"I can't... I don't understand Spanish. Your camera... did you take a picture of something? Can you show me?"
detectedLang"en"
heavyLang"en"
scores
en0.8085983510011778
tl0.6048632218844985
no0.5444989488437281
nl0.5152870991797166
ro0.48778565799842394
da0.47283049472830496
sv0.468954248366013
de0.46808510638297873
it0.4645799011532125
sq0.43722943722943725
ca0.4106980961015413
es0.3873704052780396
cs0.3803622497616778
fr0.37858508604206503
yo0.375
pt0.37076476282671833
ms0.3658536585365854
hr0.3414387031408308
sl0.34077079107505076
pl0.3222106360792492
is0.31650893796004204
hu0.3114406779661017
sk0.3092454835281615
tr0.2562929061784897
et0.2537313432835821
az0.24418604651162787
lt0.18238993710691823
fi0.1751269035532995
lv0.1698595146871009
vi0.1698595146871009
be0.09722222222222222
eu0.09722222222222222
uk0.08835904628330996
reliabletrue
passestrue
4
name"Maria"
dialogue"¡Hay una cara! En la oscuridad. ¡Es el diablo! ¡Él se llevó a las personas!"
detectedLang"es"
heavyLang"es"
scores
es0.8263888888888888
ca0.7320061255742726
ro0.5983167559296098
it0.5664739884393063
fr0.5422842197035745
tl0.5320855614973261
pt0.5075046904315197
sv0.49856733524355296
no0.4902912621359223
da0.460431654676259
hu0.4593202883625128
lv0.44620253164556967
sq0.43790149892933616
lt0.4360902255639098
de0.3958573072497123
tr0.38739789964994165
is0.38668224299065423
cs0.38380281690140844
ms0.37796208530805686
yo0.37722419928825623
hr0.3667068757539204
nl0.362089914945322
sl0.3613138686131387
vi0.3597560975609756
en0.3462017434620174
fi0.3217054263565891
sk0.2647058823529412
et0.2563739376770538
pl0.23802612481857763
eu0.2164179104477612
az0.19724770642201836
reliabletrue
passestrue
5
name"Hunter"
dialogue"I'm sorry, I have to go. I'll find someone who can translate. Wait here, or meet me at the station later? ¿Policía? ... Station. I'll come back."
detectedLang"en"
heavyLang"en"
scores
en0.8044487900268883
tl0.6144578313253012
da0.5951417004048583
no0.5719636169074371
it0.494949494949495
fr0.46380697050938335
sq0.4598244429439568
nl0.45689069925322473
pl0.4542974079126876
sv0.44095038434661077
pt0.42569992821249103
es0.41477688368690563
yo0.41348973607038125
hr0.40476190476190477
cs0.3971363978899774
de0.38080495356037153
ca0.36608557844690964
ro0.3464052287581699
sl0.3453355155482815
sk0.33993399339933994
lv0.33554817275747506
lt0.3311036789297659
et0.3133047210300429
ms0.31153184165232356
tr0.2988606485539001
az0.2963940193491645
fi0.23150816522574447
eu0.1675338189386056
vi0.1587802313354364
hu0.15074309978768577
zh0.08779931584948689
hi0.0867579908675799
be0.08045977011494253
is0.07300115874855156
uk0.07300115874855156
reliabletrue
passestrue