Run: 2026-05-10-01KR8XS1PDZWST09T1NENPF14J

Run Details

Model:
Gemma 4 31B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
2m 17s
Cost:
$0.000714
Provider:
Together

Evaluation summary

Score Evaluator
Details Link
58.33% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. Miss? I'm Detective Hunter. I'm investigating the disappearance of the Miller boy. You were sketching and taking photos near the orchard yesterday—did you see anything out of the ordinary?" Selvi: "Mujhe nahi pata aap kya keh rahe hain... lekin maine wahan kuch dekha. Kuch bahut ajeeb. Main abhi bhi kaanp rahi hoon." Hunter: "I’m sorry? I don't follow. Do you speak English? I just need to know if you saw anyone heading toward the creek around six o'clock." Selvi: "Main English nahi jaanti! Kya aap meri baat samajh nahi rahe? Dekhiye, mere camera mein... yeh dekhiye! Yeh insaan nahi tha, yeh kuch aur hi tha!" Hunter: "Look, I don't know what language that is, but you look shaken up. Just nod your head—yes or no—did you see a ten-year-old boy in a red jacket?" Selvi: "Aap bas sawal puch rahe hain, lekin meri baat suniye! Yeh bahut khatarnak hai. Humein kisi ko batana hoga, isse pehle ki koi aur gayab ho jaye!" Hunter: "I can't understand a word you're saying. Is there anyone here who can translate? A teacher? A friend?" Selvi: "Main akeli hoon! Please, mere camera ki screen dekhiye! Yahan... is ped ke peeche... woh cheez!" Hunter: "I see a blurry shadow in a photo, kid. I can't build a case on a smudge. Just stay put, I'll try to find someone who—" (Hunter's phone rings loudly. He answers it immediately.) Hunter: "Hunter here. What? Another one? Where? ...Dammit. I'm on my way." Hunter: (To Selvi) "I have to go. Stay right here. Don't move, and don't delete those photos." Selvi: "Aap kahan ja rahe hain? Meri baat suniye! Aap aise nahi ja sakte!"
created_at"2026-05-10T12:28:45.653Z" 5/10/2026, 12:28:45 PM
model"google/gemma-4-31b-it-20260402"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency687
moderation_latency(null)
generation_time137183
tokens_prompt315
tokens_completion1197
native_tokens_prompt316
native_tokens_completion1301
native_tokens_completion_images(null)
native_tokens_reasoning775
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.0007137
router(null)
provider_responses
0
endpoint_id"576fca5b-f09c-4bda-b560-9989c39ac3ab"
id"oiXYW8Z-zfFMX-9f98f2118436cb7e"
is_byokfalse
latency442
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"Together"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778416125-6PJp3DGqaq6x8SduNCsO"
session_id(null)
api_type"completions"
id"gen-1778416125-OereKMHyTMXDz7AZ4K77"
upstream_id"oiXYW8Z-zfFMX-9f98f2118436cb7e"
total_cost0.0007137
cache_discount(null)
upstream_inference_cost0
provider_name"Together"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
58.33% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. Miss? I'm Detective Hunter. I'm investigating the disappearance of the Miller boy. You were sketching and taking photos near the orchard yesterday—did you see anything out of the ordinary?"
detectedLang"en"
heavyLang"en"
scores
en0.7857403355215171
tl0.6373456790123457
fr0.5458059528411288
nl0.5256358498183286
no0.4761480160499331
de0.442863916548127
it0.4340077071290945
pt0.4139650872817955
ro0.3968172484599589
da0.38897555902236086
sq0.38190426091530777
es0.381578947368421
sv0.3672590199246096
ca0.3511871893981226
et0.3239355581127733
sl0.32041642567958356
ms0.31923522595596754
is0.27825552825552824
cs0.26054122089364373
hr0.2586750788643533
yo0.24339987121699935
tr0.2103494623655914
fi0.19740437158469945
lt0.18853591160220995
sk0.18572418572418573
az0.14978292329956583
lv0.10714285714285714
pl0.08274785323965651
hu0.07041139240506329
eu0.06374501992031872
hi0.06150159744408946
reliabletrue
passestrue
1
name"Selvi"
dialogue"Mujhe nahi pata aap kya keh rahe hain... lekin maine wahan kuch dekha. Kuch bahut ajeeb. Main abhi bhi kaanp rahi hoon."
detectedLang"eu"
heavyLang"tl"
scores
eu0.3829787234042553
fi0.32242990654205606
tl0.30488974113135187
fr0.2899118511263467
en0.26545086119554206
ms0.25103305785123964
de0.23684210526315788
et0.18355855855855857
ro0.16857798165137614
sq0.12650602409638553
is0.1082410824108241
lv0.1082410824108241
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I’m sorry? I don't follow. Do you speak English? I just need to know if you saw anyone heading toward the creek around six o'clock."
detectedLang"en"
heavyLang"en"
scores
en0.817251461988304
tl0.6249464209172739
no0.40476190476190477
pl0.37095614665708126
sv0.358974358974359
yo0.3566176470588235
da0.35185185185185186
cs0.338124054462935
hr0.338124054462935
sk0.3180046765393608
nl0.2988782051282051
ms0.28687856560717195
sq0.2732558139534884
et0.2578456318914334
it0.2508561643835616
pt0.22634836427939875
sl0.21594982078853048
ca0.19724770642201836
fr0.1784037558685446
lv0.17452830188679244
vi0.16587225929456625
lt0.1602687140115163
ro0.1562198649951784
is0.13366336633663367
es0.11347517730496455
de0.07602956705385427
be0.07407407407407407
tr0.07407407407407407
hu0.07112526539278131
az0.06716417910447761
uk0.06716417910447761
reliabletrue
passestrue
3
name"Selvi"
dialogue"Main English nahi jaanti! Kya aap meri baat samajh nahi rahe? Dekhiye, mere camera mein... yeh dekhiye! Yeh insaan nahi tha, yeh kuch aur hi tha!"
detectedLang"ms"
heavyLang"tl"
scores
ms0.42731277533039647
tl0.3890977443609023
ro0.3821292775665399
en0.3737957610789981
de0.34804413239719156
et0.34343434343434337
da0.3271221532091097
sv0.3236212278876171
ca0.2911668484187568
it0.2911668484187568
fi0.24855491329479767
sq0.22062350119904073
tr0.22062350119904073
no0.21875
lv0.20924574209245742
eu0.20245398773006135
fr0.20147420147420148
hr0.1761723700887199
az0.15803108808290156
is0.1356382978723404
es0.1321762349799733
lt0.11924119241192413
nl0.10468319559228649
cs0.09344490934449094
sk0.09344490934449094
sl0.09344490934449094
reliablefalse
passesfalse
4
name"Hunter"
dialogue"Look, I don't know what language that is, but you look shaken up. Just nod your head—yes or no—did you see a ten-year-old boy in a red jacket?"
detectedLang"en"
heavyLang"en"
scores
en0.8211279461279463
tl0.66
nl0.47335811648079307
sv0.4221617946974847
et0.3988684582743989
ro0.3963068181818182
no0.371301775147929
pl0.36990363232023726
ca0.36377245508982037
yo0.35945742275810094
it0.3545937737281701
hr0.3343774471417384
cs0.3275316455696203
sk0.3156199677938808
hu0.30725346373268136
da0.30498773507767785
es0.29343308395677475
fr0.29343308395677475
sl0.2728828058169376
is0.2697594501718213
lt0.26279271465741544
ms0.26279271465741544
de0.26151172893136404
pt0.24645390070921988
lv0.236983842010772
tr0.21658986175115208
az0.20337394564198688
sq0.19507575757575757
eu0.14829659318637273
vi0.1353001017293998
be0.07608695652173912
fi0.06900328587075576
uk0.06900328587075576
reliabletrue
passestrue
5
name"Selvi"
dialogue"Aap bas sawal puch rahe hain, lekin meri baat suniye! Yeh bahut khatarnak hai. Humein kisi ko batana hoga, isse pehle ki koi aur gayab ho jaye!"
detectedLang"eu"
heavyLang"ber"
scores
eu0.3425728500355366
sl0.3163340724316335
ms0.281832298136646
tl0.26761678543151224
it0.23044925124792015
ca0.2213804713804714
yo0.20872540633019673
hu0.20464316423043852
az0.2032730404823428
tr0.18430335097001763
fr0.1659152389540126
ro0.1606170598911071
fi0.1583257506824386
hr0.14667896678966788
vi0.14510166358595195
es0.13389513108614234
pt0.13389513108614234
lv0.11736641221374046
no0.10714285714285714
sk0.10714285714285714
cs0.10281280310378274
et0.10194174757281553
sq0.10194174757281553
lt0.09313725490196079
is0.08687068114511352
sv0.0777666999002991
reliablefalse
passesfalse
6
name"Hunter"
dialogue"I can't understand a word you're saying. Is there anyone here who can translate? A teacher? A friend?"
detectedLang"en"
heavyLang"en"
scores
en0.8172712506767731
tl0.5523872679045093
nl0.5188880969351389
de0.5136887608069165
no0.490566037735849
da0.48940998487140697
sv0.452998379254457
ro0.41048034934497823
sq0.40318302387267907
it0.39999999999999997
ca0.3869209809264305
hr0.3656015037593985
tr0.3644067796610169
cs0.36018957345971564
pl0.35958254269449713
hu0.3583650190114069
sk0.3440233236151603
yo0.32567432567432564
is0.3105209397344229
fr0.30268595041322316
az0.2849576271186441
vi0.24665178571428573
pt0.24581005586592178
lt0.23900789177001125
es0.23642533936651583
lv0.2285714285714286
et0.21511627906976746
sl0.18082524271844658
eu0.1687192118226601
ms0.15307402760351316
be0.09395973154362416
fi0.08536585365853659
uk0.08536585365853659
reliabletrue
passestrue
7
name"Selvi"
dialogue"Main akeli hoon! Please, mere camera ki screen dekhiye! Yahan... is ped ke peeche... woh cheez!"
detectedLang"en"
heavyLang"en"
scores
en0.5934959349593496
tl0.4434137291280148
ro0.37694704049844235
ms0.3684210526315789
nl0.3548387096774194
yo0.3485342019543974
hu0.3142857142857143
es0.2248062015503876
fi0.2227979274611399
fr0.21465968586387435
da0.20844327176781002
it0.20844327176781002
az0.20634920634920634
sl0.20318725099601595
de0.2
tr0.14772727272727273
cs0.1465149359886202
ca0.1452991452991453
is0.13294797687861273
sv0.12023460410557184
no0.1111111111111111
reliabletrue
passesfalse
8
name"Hunter"
dialogue"I see a blurry shadow in a photo, kid. I can't build a case on a smudge. Just stay put, I'll try to find someone who—"
detectedLang"en"
heavyLang"en"
scores
en0.7794341344691208
tl0.5137491616364857
hr0.4444444444444444
et0.4322631166797181
pl0.42414614773629866
cs0.42277070063694266
yo0.41104792851340366
it0.3902439024390244
ca0.3829787234042553
ro0.38192668371696503
fr0.36459246275197194
da0.3640350877192982
sv0.36123348017621143
sk0.36067019400352734
sl0.3348623853211009
nl0.3101807802093245
no0.30886558627264066
lv0.2602040816326531
lt0.25564681724846
hu0.2352320675105485
pt0.23117709437963943
az0.23036093418259024
de0.22790202342917998
fi0.22707889125799574
es0.22459893048128343
sq0.22126745435016112
hi0.18171557562076748
tr0.16857798165137614
is0.15990730011587484
ms0.15990730011587484
vi0.15501165501165504
be0.0880503144654088
eu0.0880503144654088
uk0.0799492385786802
reliabletrue
passestrue
9
name"Hunter"
dialogue"Hunter here. What? Another one? Where? ...Dammit. I'm on my way."
detectedLang"en"
heavyLang"en"
scores
en0.8212583412774069
tl0.5241116751269036
pl0.41222570532915365
sq0.3872549019607844
sv0.336283185840708
fr0.32553956834532377
hr0.3181818181818182
de0.30683918669131244
cs0.30037313432835816
sk0.30037313432835816
da0.29906542056074764
ca0.2964352720450282
et0.29111531190926276
fi0.2857142857142857
es0.21052631578947367
hi0.1830065359477124
no0.17218543046357618
yo0.16666666666666666
it0.14383561643835618
nl0.14383561643835618
pt0.14383561643835618
ms0.1359447004608295
ro0.1359447004608295
sl0.1359447004608295
tr0.1359447004608295
reliabletrue
passestrue
10
name"Hunter"
dialogue"I have to go. Stay right here. Don't move, and don't delete those photos."
detectedLang"en"
heavyLang"en"
scores
en0.8259860788863108
tl0.5829471733086191
da0.5412844036697247
no0.5197438633938101
yo0.512987012987013
sq0.4712103407755582
pl0.45054945054945056
fr0.4498777506112469
ca0.38016528925619836
lv0.3626062322946175
hr0.3617021276595745
lt0.35437589670014347
cs0.3440233236151603
ro0.3401759530791789
sv0.31297709923664124
sk0.3119266055045872
sl0.303405572755418
it0.2879746835443038
pt0.26229508196721313
hu0.25124792013311154
es0.24623115577889448
de0.24369747899159663
tr0.24369747899159663
et0.23469387755102045
nl0.23469387755102045
hi0.14611005692599618
be0.1346153846153846
az0.12280701754385964
is0.12280701754385964
uk0.12280701754385964
vi0.12280701754385964
reliabletrue
passestrue
11
name"Selvi"
dialogue"Aap kahan ja rahe hain? Meri baat suniye! Aap aise nahi ja sakte!"
detectedLang"eu"
heavyLang"tl"
scores
eu0.4640122511485452
tl0.44178628389154706
et0.42528735632183906
sq0.42244224422442245
fi0.4077834179357022
de0.33962264150943394
da0.308300395256917
ms0.2827868852459016
lv0.24731182795698922
ca0.2374727668845316
tr0.23076923076923078
nl0.2222222222222222
sk0.1935483870967742
pl0.18604651162790697
sl0.18604651162790697
no0.1784037558685446
sv0.1764705882352941
yo0.1764705882352941
reliablefalse
passesfalse