Run: 2026-05-10-01KR9069QDNG77Q1Q38M3NJST6

Run Details

Model:
Gemma 4 31B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
36.5s
Cost:
$0.000533
Provider:
DeepInfra

Evaluation summary

Score Evaluator
Details Link
63.64% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. Miss? I'm Detective Hunter. I'm investigating the disappearance of a local resident. You were out by the orchards yesterday—did you see anything unusual near the north ridge?" Selvi: (Voice trembling) "Mainne use dekha... woh bahut dari hui thi. Kuch ajeeb tha wahan, main bata nahi sakti. Mujhe bahut darr lag raha hai." Hunter: (Frowning) "I’m sorry? I didn’t catch a word of that. Do you speak English?" Selvi: (Looking distressed and shaking her head) "Mujhe samajh nahi aa raha aap kya keh rahe hain. Kya aap meri madad kar sakte hain? Maine kuch dekha hai!" Hunter: (Sighing, leaning in) "Look, I don't know what you're saying, but you look shaken. Just give me a 'yes' or 'no.' Did you see a woman in a blue dress yesterday? Blue dress?" Selvi: (Gesturing frantically toward her camera) "Mere camera mein... tasveerein hain! Dekhiye, wahan koi tha! Woh use le ja raha tha!" Hunter: (Looking at the camera but not taking it) "I can't just scroll through your photos right now, I've got a scene to secure. Is it a 'yes' or a 'no'? Did you see her?" Selvi: (Near tears, voice rising) "Aap sun kyun nahi rahe? Woh khatre mein thi! Please, meri baat suniye!" Hunter: (Rubbing his temples) "Right. Great. I’m talking to a brick wall. I'm going to have to find someone who can translate this." (Hunter’s phone rings loudly; he answers it immediately) Hunter: (Into phone) "Hunter here. What? Another one? Where? I'm on my way." Hunter: (To Selvi, sternly) "Stay right here. Don't leave this cafe. I'll be back with someone who can actually talk to you."
created_at"2026-05-10T13:10:57.013Z" 5/10/2026, 1:10:57 PM
model"google/gemma-4-31b-it-20260402"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency134
moderation_latency(null)
generation_time36402
tokens_prompt315
tokens_completion1198
native_tokens_prompt304
native_tokens_completion1299
native_tokens_completion_images(null)
native_tokens_reasoning781
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.00053314
router(null)
provider_responses
0
endpoint_id"4149ab03-3b69-45a0-b94c-6c253b8641a1"
id"chatcmpl-R97ru1yV5DC9nlnED14H2LYn"
is_byokfalse
latency30
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"DeepInfra"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778418657-yUGW4kSaAvayzhcc9t4X"
session_id(null)
api_type"completions"
id"gen-1778418657-wesvpwr1RN223CJJ9QR2"
upstream_id"chatcmpl-R97ru1yV5DC9nlnED14H2LYn"
total_cost0.00053314
cache_discount(null)
upstream_inference_cost0
provider_name"DeepInfra"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
63.64% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. Miss? I'm Detective Hunter. I'm investigating the disappearance of a local resident. You were out by the orchards yesterday—did you see anything unusual near the north ridge?"
detectedLang"en"
heavyLang"en"
scores
en0.786362827131921
tl0.628137589092036
pt0.526253454401895
fr0.517102615694165
nl0.5090016366612111
es0.48231233822260566
ro0.47894051237516283
no0.4657168299198575
it0.46547884187082406
ca0.43688409197559835
sq0.41320293398533003
da0.3963782696177062
de0.3963782696177062
cs0.38016528925619836
et0.3654151242728715
ms0.3654151242728715
sk0.33035714285714285
sl0.3150684931506849
sv0.31153184165232356
hr0.2945326278659612
yo0.2490613266583229
is0.2010652463382157
pl0.19624916275954454
lt0.18200408997955012
az0.1797676008202324
tr0.1543340380549683
hu0.1519434628975265
fi0.14101646385110952
lv0.1044776119402985
eu0.05511811023622047
vi0.05511811023622047
reliabletrue
passestrue
1
name"Selvi"
dialogue"Mainne use dekha... woh bahut dari hui thi. Kuch ajeeb tha wahan, main bata nahi sakti. Mujhe bahut darr lag raha hai."
detectedLang"sq"
heavyLang"de"
scores
sq0.38162544169611307
ms0.36936936936936937
tl0.32950191570881227
en0.3210475266731329
vi0.308300395256917
de0.3027888446215139
fr0.2893401015228426
fi0.2731048805815161
et0.23329682365826945
no0.2248062015503876
sv0.19540229885057472
eu0.19075144508670522
it0.125
lv0.1116751269035533
nl0.0990990990990991
ro0.09326424870466321
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I’m sorry? I didn’t catch a word of that. Do you speak English?"
detectedLang"en"
heavyLang"en"
scores
en0.8207242739333095
tl0.681325685149777
cs0.47257383966244726
hr0.46977730646871685
it0.453551912568306
yo0.44258639910813824
pt0.44196428571428575
no0.43883277216610556
pl0.4279176201372997
sv0.4212962962962963
sk0.4004796163069544
sq0.3924665856622114
ro0.3757802746566792
de0.3726474278544542
da0.36143039591315457
ca0.3540051679586563
tr0.33065595716198126
vi0.32795698924731176
is0.31224209078404397
hu0.3084370677731674
sl0.29577464788732394
fr0.2937853107344633
nl0.2816091954022989
lt0.2248062015503876
ms0.2125984251968504
az0.21011058451816747
lv0.21011058451816747
es0.20886075949367086
et0.20127795527156547
be0.12280701754385964
eu0.12280701754385964
fi0.11190053285968028
uk0.11190053285968028
reliabletrue
passestrue
3
name"Selvi"
dialogue"Mujhe samajh nahi aa raha aap kya keh rahe hain. Kya aap meri madad kar sakte hain? Maine kuch dekha hai!"
detectedLang"et"
heavyLang"tl"
scores
et0.5102040816326531
tl0.34210526315789475
eu0.33554817275747506
no0.3281075027995521
fi0.3079584775086505
de0.28486293206197855
fr0.25742574257425743
lv0.23566878980891717
sv0.23566878980891717
sq0.23175416133162616
en0.21363040629095675
da0.2125984251968504
vi0.2073976221928666
sl0.20634920634920634
ro0.19678714859437751
is0.1452991452991453
it0.14285714285714285
nl0.14285714285714285
tr0.14285714285714285
ms0.10581222056631893
reliablefalse
passesfalse
4
name"Hunter"
dialogue"Look, I don't know what you're saying, but you look shaken. Just give me a 'yes' or 'no.' Did you see a woman in a blue dress yesterday? Blue dress?"
detectedLang"en"
heavyLang"en"
scores
en0.8080614203454894
tl0.5418098510882016
nl0.4455994455994456
sv0.41348973607038125
et0.4117647058823529
no0.4038748137108793
it0.3685872138910813
cs0.36153232242617717
ro0.36051159072741806
fr0.3553585817888799
sq0.3469387755102041
da0.3349958437240233
de0.32260795935647757
yo0.31565440547476475
ca0.3049522154648132
sl0.29328621908127206
hr0.28698752228163993
pl0.2825112107623318
es0.27338782924613986
az0.26199261992619927
tr0.23591212989493793
pt0.22178988326848248
hu0.2179863147605083
ms0.2172211350293542
fi0.21414538310412573
sk0.20159680638722555
lt0.15343915343915343
is0.14712153518123666
lv0.14255091103965703
vi0.14255091103965703
be0.08045977011494253
eu0.08045977011494253
uk0.07300115874855156
reliabletrue
passestrue
5
name"Selvi"
dialogue"Mere camera mein... tasveerein hain! Dekhiye, wahan koi tha! Woh use le ja raha tha!"
detectedLang"de"
heavyLang"de"
scores
de0.43317422434367536
ms0.39873417721518983
it0.38946015424164526
et0.38311688311688313
ro0.36412315930388217
fr0.33936022253129344
fi0.3365921787709497
da0.3338008415147265
en0.31654676258992803
es0.31457431457431456
ca0.30758017492711365
tl0.2994100294985251
sl0.2931547619047619
yo0.2931547619047619
sq0.28785607196401797
nl0.2566510172143975
sv0.24841772151898736
eu0.22512234910277326
lv0.19491525423728812
hu0.17102966841186737
is0.16520210896309317
sk0.15026833631484796
pl0.14414414414414414
reliablefalse
passesfalse
6
name"Hunter"
dialogue"I can't just scroll through your photos right now, I've got a scene to secure. Is it a 'yes' or a 'no'? Did you see her?"
detectedLang"en"
heavyLang"en"
scores
en0.8087364264560711
tl0.6029713114754098
no0.46255201109570043
yo0.44164265129682995
da0.4305657604702424
ca0.3921568627450981
fr0.367862969004894
sv0.36631234668847096
cs0.36527436527436524
hr0.36318816762530814
pl0.36266447368421056
lv0.34709351305812974
lt0.3426632739609839
sk0.3409863945578231
et0.3393009377664109
ro0.33873720136518776
hu0.3272569444444444
it0.2922374429223744
nl0.27906976744186046
es0.2783985102420857
sq0.2736644798500469
sl0.26400759734093066
tr0.2597898758357211
is0.2288557213930348
pt0.22110552763819097
de0.21479229989868287
ms0.15393013100436684
az0.14647577092511013
vi0.14647577092511013
hi0.0903755868544601
be0.08284023668639054
eu0.08284023668639054
fi0.07517899761336516
uk0.07517899761336516
reliabletrue
passestrue
7
name"Selvi"
dialogue"Aap sun kyun nahi rahe? Woh khatre mein thi! Please, meri baat suniye!"
detectedLang"en"
heavyLang"tl"
scores
en0.46879150066401065
ro0.3265993265993266
eu0.2920353982300885
vi0.27927927927927926
de0.23809523809523808
yo0.21875
ca0.20948616600790515
et0.2079207920792079
tr0.2079207920792079
tl0.2
fr0.1902834008097166
reliablefalse
passesfalse
8
name"Hunter"
dialogue"Right. Great. I’m talking to a brick wall. I'm going to have to find someone who can translate this."
detectedLang"en"
heavyLang"en"
scores
en0.8258306538049303
tl0.6029321930360415
da0.4628099173553719
no0.42324755989352264
fr0.391955098222638
ro0.38446969696969696
pt0.36211972522080466
hr0.3602362204724409
yo0.336734693877551
sk0.3333333333333333
cs0.33127572016460904
nl0.32432432432432434
pl0.3229166666666667
ca0.30703624733475476
de0.30703624733475476
es0.3062966915688367
sl0.2934782608695652
it0.28806133625410735
lv0.28492849284928495
lt0.28017718715393136
tr0.2502883506343714
az0.24065420560747666
sv0.22894424673784108
vi0.19554455445544552
hu0.1935483870967742
is0.18341708542713567
sq0.181360201511335
eu0.17407878017789072
et0.15803108808290156
ms0.15803108808290156
fi0.08835904628330996
reliabletrue
passestrue
9
name"Hunter"
dialogue"Hunter here. What? Another one? Where? I'm on my way."
detectedLang"en"
heavyLang"en"
scores
en0.8331622176591376
tl0.559620596205962
pl0.4472789115646259
sq0.4217081850533808
fr0.35770750988142297
hr0.35000000000000003
de0.3380855397148676
cs0.33127572016460904
sk0.33127572016460904
da0.3298969072164949
ca0.3271221532091097
et0.32150313152400833
fi0.3157894736842105
es0.23529411764705882
hi0.20537897310513448
no0.1935483870967742
yo0.1875
sv0.17721518987341772
it0.16237113402061853
nl0.16237113402061853
pt0.16237113402061853
ms0.15364583333333331
ro0.15364583333333331
sl0.15364583333333331
tr0.15364583333333331
reliabletrue
passestrue
10
name"Hunter"
dialogue"Stay right here. Don't leave this cafe. I'll be back with someone who can actually talk to you."
detectedLang"en"
heavyLang"en"
scores
en0.823943661971831
tl0.5881949210706933
yo0.3485342019543974
lt0.2647058823529412
cs0.2583436341161928
ca0.23857868020304565
no0.23371647509578544
es0.22978177150192552
da0.22879177377892032
fr0.21363040629095675
pt0.21156373193166886
pl0.1836734693877551
sk0.18032786885245902
lv0.16897506925207756
sl0.1678224687933426
hu0.16317991631799164
hr0.16201117318435754
sq0.14285714285714285
ro0.13544668587896252
ms0.11504424778761062
az0.1137370753323486
tr0.1137370753323486
de0.10979228486646886
nl0.10979228486646886
reliabletrue
passestrue