Run: 2026-05-10-01KR92DMSXTKJSAM2MAPZGB29X

Run Details

Model:
Gemma 4 31B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
3m 25s
Cost:
$0.000596
Provider:
Together

Evaluation summary

Score Evaluator
Details Link
63.64% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. Miss? I'm Detective Hunter. I'm talking to everyone who was in the square this morning. Did you see anything unusual near the old mill around 10:00 AM?" Selvi: (Breathless and shaking) "Maine dekha! Maine unhe dekha! Woh bahut dare hue the aur kisi ka peecha kar rahe the... mujhe samajh nahi aa raha ki kya ho raha tha." Hunter: (Frowning) "I’m sorry? Do you speak English?" Selvi: (Urgent, pointing to her camera) "Yeh dekhiye! Mere camera mein dekhiye! Maine ek photo li thi, par woh saaf nahi hai. Par woh wahan the, main kasam kha sakti hoon!" Hunter: "I have no idea what you're saying. Is that a camera? Are you trying to show me a photo?" Selvi: (Frantic, gesturing toward the woods) "Wahan! Jungle ki taraf! Woh unhe le gaye! Please, meri baat suniye!" Hunter: (Sighing, rubbing his temple) "Look, I can see you're upset, but I can't understand a word. Can you point to a map? Or maybe write it down?" Selvi: (Tearfully) "Main kaise bataun? Mujhe English nahi aati! Woh khatre mein hain!" Hunter: "Right. No English. Look, just stay right here. I’ll find someone who can translate—" (Hunter's phone rings loudly) Hunter: "Dammit. (Into phone) Hunter here. What? Now? I'm on my way." Hunter: (To Selvi) "I have to go. Don't leave this cafe. I'll be back for you."
created_at"2026-05-10T13:50:01.451Z" 5/10/2026, 1:50:01 PM
model"google/gemma-4-31b-it-20260402"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency5978
moderation_latency(null)
generation_time198175
tokens_prompt315
tokens_completion981
native_tokens_prompt316
native_tokens_completion1065
native_tokens_completion_images(null)
native_tokens_reasoning636
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.0005957
router(null)
provider_responses
0
endpoint_id"4149ab03-3b69-45a0-b94c-6c253b8641a1"
is_byokfalse
latency110
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"DeepInfra"
status429
1
endpoint_id"29b94af5-d8f8-419a-9cf1-ae1f2831142b"
is_byokfalse
latency110
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"Venice"
status429
2
endpoint_id"576fca5b-f09c-4bda-b560-9989c39ac3ab"
id"oiXyVaH-4nba6j-9f99691ca5220432"
is_byokfalse
latency5502
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"Together"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778421001-25sSTqjxTLHno8vJvcjT"
session_id(null)
api_type"completions"
id"gen-1778421001-KUOlW5BARdbNDLvbP16C"
upstream_id"oiXyVaH-4nba6j-9f99691ca5220432"
total_cost0.0005957
cache_discount(null)
upstream_inference_cost0
provider_name"Together"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
63.64% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. Miss? I'm Detective Hunter. I'm talking to everyone who was in the square this morning. Did you see anything unusual near the old mill around 10:00 AM?"
detectedLang"en"
heavyLang"en"
scores
en0.8113755078351711
tl0.6159905474596298
nl0.45561139028475717
ro0.41016333938294003
no0.40403422982885084
it0.4025735294117647
de0.3996305418719212
sv0.3517287234042553
fr0.34121621621621623
sl0.3340163934426229
da0.3298969072164949
sq0.3289745354439092
pt0.324792243767313
ms0.3162692847124825
yo0.30057388809182206
et0.275092936802974
es0.268018018018018
cs0.22372611464968153
ca0.1968698517298188
pl0.18341708542713567
hr0.15803108808290156
is0.15657439446366783
sk0.11924119241192413
lv0.11121239744758432
lt0.09638554216867469
hi0.08192090395480227
az0.07932011331444759
fi0.07142857142857142
tr0.05705996131528047
reliabletrue
passestrue
1
name"Selvi"
dialogue"Maine dekha! Maine unhe dekha! Woh bahut dare hue the aur kisi ka peecha kar rahe the... mujhe samajh nahi aa raha ki kya ho raha tha."
detectedLang"et"
heavyLang"de"
scores
et0.4318181818181818
sq0.37928082191780826
no0.36846689895470386
tl0.35325602140945583
en0.32369402985074625
lv0.318609022556391
yo0.3108365019011407
sl0.2988394584139265
fr0.2828882294757666
fi0.2691532258064516
ro0.2579324462640737
it0.2352320675105485
tr0.21959095801937567
pl0.20242024202420242
eu0.18539325842696627
da0.1844769403824522
sv0.17893544733861835
az0.17707150964812712
ms0.17048054919908467
de0.16857798165137614
es0.14705882352941177
hu0.1430260047281324
ca0.14201183431952663
sk0.13277511961722488
cs0.12755716004813478
is0.12333736396614269
nl0.103831891223733
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I’m sorry? Do you speak English?"
detectedLang"en"
heavyLang"en"
scores
en0.8006379585326954
tl0.6565934065934066
pt0.5059288537549407
cs0.4577006507592191
hr0.43438914027149317
pl0.36386768447837153
sk0.34554973821989526
sl0.32432432432432434
sq0.31880108991825606
vi0.30747922437673125
it0.2857142857142857
no0.24698795180722888
yo0.2378048780487805
da0.22360248447204967
de0.22360248447204967
ms0.22360248447204967
ro0.22360248447204967
sv0.22360248447204967
reliabletrue
passestrue
3
name"Selvi"
dialogue"Yeh dekhiye! Mere camera mein dekhiye! Maine ek photo li thi, par woh saaf nahi hai. Par woh wahan the, main kasam kha sakti hoon!"
detectedLang"fr"
heavyLang"de"
scores
fr0.5195493704440026
en0.4887165021156558
tl0.46256486286137877
de0.3679163034001744
ms0.35152057245080504
ro0.34329710144927533
it0.3198874296435272
da0.31214421252371916
vi0.3008678881388621
sq0.28430404738400794
nl0.2639593908629441
sv0.25869120654396727
fi0.257172131147541
lv0.25103305785123964
ca0.22707889125799574
tr0.22625400213447172
cs0.19084821428571427
eu0.18539325842696627
et0.18355855855855857
yo0.12860576923076922
hr0.12227602905569007
es0.12014563106796115
no0.11585365853658537
hi0.09600997506234414
lt0.09261576971214018
pl0.09261576971214018
reliablefalse
passesfalse
4
name"Hunter"
dialogue"I have no idea what you're saying. Is that a camera? Are you trying to show me a photo?"
detectedLang"en"
heavyLang"en"
scores
en0.8224087826929287
tl0.6054519368723099
ca0.5166959578207382
it0.5036101083032491
es0.5031616982836495
ro0.49308755760368667
cs0.492619926199262
no0.4821092278719397
fr0.462890625
hr0.45972495088408644
nl0.4586614173228346
yo0.4570582428430405
da0.45652173913043476
pt0.44612286002014095
pl0.44500504540867813
sk0.4358974358974359
lv0.4123931623931624
hu0.4066882416396979
sq0.39159292035398224
tr0.3785310734463277
sl0.35217903415783275
ms0.34988179669030733
sv0.33973589435774315
lt0.311639549436796
is0.2948717948717949
et0.28664072632944226
de0.20174165457184323
fi0.2005813953488372
az0.19472913616398244
vi0.19472913616398244
hi0.12280701754385964
be0.11290322580645161
eu0.11290322580645161
uk0.10277324632952692
reliabletrue
passestrue
5
name"Selvi"
dialogue"Wahan! Jungle ki taraf! Woh unhe le gaye! Please, meri baat suniye!"
detectedLang"ms"
heavyLang"de"
scores
ms0.5046620046620046
tr0.5046620046620046
en0.44948186528497414
tl0.4209809264305177
ro0.4072524407252441
sl0.38848920863309355
fr0.36661698956780925
yo0.356060606060606
hu0.34006211180124224
az0.2685025817555938
it0.25438596491228066
de0.22727272727272727
es0.21875
lt0.2085661080074488
et0.19811320754716982
fi0.17153996101364524
reliablefalse
passesfalse
6
name"Hunter"
dialogue"Look, I can see you're upset, but I can't understand a word. Can you point to a map? Or maybe write it down?"
detectedLang"en"
heavyLang"en"
scores
en0.811726734803658
tl0.5370370370370371
da0.5234853641933288
no0.4700984102952308
fr0.44400317712470216
pl0.4336569579288026
cs0.4308943089430894
sk0.4190871369294606
yo0.3744414655942806
sv0.37050359712230213
ro0.35838680109990834
et0.3566176470588235
hr0.3439550140581068
ca0.33396764985727884
lv0.3275696445725264
lt0.3143976493633692
it0.308300395256917
sq0.29435483870967744
sl0.28789420142421157
de0.27083333333333337
nl0.26160337552742613
tr0.24242424242424243
hu0.24160346695557963
is0.2324561403508772
az0.23076923076923078
fi0.1935483870967742
pt0.16666666666666666
vi0.15966386554621848
es0.15865384615384615
be0.09090909090909091
eu0.09090909090909091
ms0.08256880733944955
uk0.08256880733944955
reliabletrue
passestrue
7
name"Selvi"
dialogue"Main kaise bataun? Mujhe English nahi aati! Woh khatre mein hain!"
detectedLang"eu"
heavyLang"tl"
scores
eu0.4968553459119497
en0.4936708860759494
fr0.47575360419397117
de0.4074074074074074
ro0.3730407523510972
ca0.3079584775086505
ms0.2647058823529412
fi0.2079207920792079
tl0.20634920634920634
no0.17012448132780084
et0.16317991631799164
da0.15254237288135594
es0.15254237288135594
hr0.15254237288135594
pt0.15254237288135594
sv0.15254237288135594
reliablefalse
passesfalse
8
name"Hunter"
dialogue"Right. No English. Look, just stay right here. I’ll find someone who can translate—"
detectedLang"en"
heavyLang"en"
scores
en0.819597417394607
tl0.5221327967806841
no0.45150115473441105
ca0.433849821215733
hr0.39567430025445294
da0.39413265306122447
pt0.38391699092088194
es0.3807040417209909
ro0.33473389355742295
sv0.3290960451977401
tr0.32432432432432434
it0.31457431457431456
de0.309593023255814
lv0.29420505200594355
sq0.2658423493044822
et0.2531446540880503
fr0.2448330683624801
sk0.2448330683624801
pl0.23387096774193547
az0.22258592471358432
ms0.21617161716171615
vi0.15630550621669628
yo0.13479052823315119
eu0.12361623616236161
lt0.12361623616236161
nl0.1171003717472119
cs0.11048689138576778
sl0.11048689138576778
reliabletrue
passestrue
9
name"Hunter"
dialogue"Dammit. Hunter here. What? Now? I'm on my way."
detectedLang"en"
heavyLang"en"
scores
en0.8127978216473791
tl0.45219123505976094
sv0.40860215053763443
fr0.39692982456140347
pl0.3902439024390244
cs0.3692660550458715
sk0.3692660550458715
sq0.3663594470046083
ca0.3648960739030023
et0.358974358974359
fi0.3529411764705882
de0.24242424242424243
hr0.24242424242424243
hi0.233983286908078
da0.22969187675070027
no0.22096317280453256
yo0.21428571428571427
it0.1863905325443787
nl0.1863905325443787
pt0.1863905325443787
ms0.17664670658682635
ro0.17664670658682635
sl0.17664670658682635
tr0.17664670658682635
reliabletrue
passestrue
10
name"Hunter"
dialogue"I have to go. Don't leave this cafe. I'll be back for you."
detectedLang"en"
heavyLang"en"
scores
en0.8214285714285714
tl0.6212121212121212
da0.5941558441558442
no0.5235069885641677
yo0.5229007633587787
pl0.4959677419354839
cs0.455732946298984
lt0.42660550458715596
hr0.40476190476190477
sk0.35233160621761656
lv0.33035714285714285
hu0.32915921288014316
ca0.2690058479532163
sv0.2690058479532163
sq0.2632612966601179
sl0.24395161290322584
it0.23469387755102045
ro0.1935483870967742
pt0.18655097613882862
be0.15730337078651685
tr0.15730337078651685
az0.14383561643835618
et0.14383561643835618
is0.14383561643835618
uk0.14383561643835618
vi0.14383561643835618
reliabletrue
passestrue