Run: 2026-05-10-01KR8ZVZDRQVYWDP564WG8XW6G

Run Details

Model:
Gemma 4 26B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
1m 2s
Cost:
$0.001264
Provider:
Novita

Evaluation summary

Score Evaluator
Details Link
58.33% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. Miss? I'm Detective Hunter. I'm looking into the disappearances near the orchards. Were you out near the apple festival grounds last night?" Selvi: "Main... main wahan thi. Maine kuch aisa dekha jo mujhe bhulane nahi de raha. Sab kuch itna ajeeb tha... main abhi bhi darr rahi hoon." *(I... I was there. I saw something that I can't forget. Everything was so strange... I'm still shaking.)* Hunter: "I’m sorry, I didn't catch any of that. Do you speak English? I need to know if you witnessed anything unusual during the festival." Selvi: "Mujhe maaf kijiye, mujhe angrezi nahi aati. Par maine wahan... wahan ek parchhayi dekhi. Camera mein... dekhiye!" *(Forgive me, I don't know English. But I saw... a shadow there. In the camera... look!)* Hunter: "A shadow? You saw a shadow? Hold on, let me see that... wait, is that a figure by the treeline in this shot?" Selvi: "Haan! Wahi! Woh bahut tezi se chala gaya." *(Yes! That! He moved very quickly.)* Hunter: "Okay, okay... let me try to get this through the translator. So, you saw a figure moving fast? Did you get a clear look at a face?" Selvi: "Koshish ki, par andhera bahut tha. Main bahut ghabraayi hui hoon." *(I tried, but it was very dark. I am very nervous.)* Hunter: "I understand. Just keep those files safe. I'll need to bring a forensics team down here to—" *[Sound of a phone ringing]* Hunter: "Hang on, I have to take this. It's the station. Stay right here." Selvi: "Kya hua? Aap ja rahe hain? Mujhe madad chahiye!" *(What happened? Are you leaving? I need help!)* Hunter: "I'll be back, I promise. Don't go anywhere."
created_at"2026-05-10T13:05:18.784Z" 5/10/2026, 1:05:18 PM
model"google/gemma-4-26b-a4b-it-20260403"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency281
moderation_latency(null)
generation_time61995
tokens_prompt315
tokens_completion2911
native_tokens_prompt304
native_tokens_completion3062
native_tokens_completion_images(null)
native_tokens_reasoning2486
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.00126432
router(null)
provider_responses
0
endpoint_id"03b96766-ef7e-454f-9519-c76b70654b49"
id"90fbb6b59019f97f164ca40336733617"
is_byokfalse
latency226
model_permaslug"google/gemma-4-26b-a4b-it-20260403"
provider_name"Novita"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778418318-BuxPwz2XxVlfIcI1MDF0"
session_id(null)
api_type"completions"
id"gen-1778418318-uuKairsP9xXoPW3lTyqQ"
upstream_id"90fbb6b59019f97f164ca40336733617"
total_cost0.00126432
cache_discount(null)
upstream_inference_cost0
provider_name"Novita"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
58.33% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. Miss? I'm Detective Hunter. I'm looking into the disappearances near the orchards. Were you out near the apple festival grounds last night?"
detectedLang"en"
heavyLang"en"
scores
en0.783289188538406
tl0.6384089995982322
fr0.6150556030795552
nl0.4874715261958998
no0.4597839135654262
pt0.4558645707376058
es0.43890274314214467
it0.4318181818181818
de0.4090610636900854
ca0.39839572192513373
da0.39189189189189183
sv0.3818681318681319
sq0.36079545454545453
ms0.3562231759656652
ro0.3227990970654628
sl0.3082244427363567
cs0.2724333063864187
et0.24623115577889448
hr0.22945205479452058
yo0.16743755781683625
fi0.13294797687861273
lv0.13211186113789777
lt0.12280701754385964
az0.11591355599214145
sk0.11591355599214145
is0.07312049433573636
hu0.06928645294725957
pl0.06542056074766354
tr0.06152241918665276
reliabletrue
passestrue
1
name"Selvi"
dialogue"Main... main wahan thi. Maine kuch aisa dekha jo mujhe bhulane nahi de raha. Sab kuch itna ajeeb tha... main abhi bhi darr rahi hoon."
detectedLang"fr"
heavyLang"tl"
scores
fr0.38917975567190227
et0.3676603432700994
en0.36536718041704447
fi0.34024505183788883
eu0.31305201177625125
ms0.30417495029821073
ro0.29718875502008035
de0.29435483870967744
tl0.29149797570850206
no0.28205128205128205
sq0.28205128205128205
ca0.25611052072263546
da0.23995656894679696
lv0.2265193370165746
yo0.20993227990970656
vi0.18128654970760233
es0.18032786885245902
nl0.17936694021101993
pt0.17550058892815076
sv0.14004914004914004
lt0.13580246913580246
sl0.1315136476426799
tr0.12280701754385964
hu0.12170639899623588
it0.08735332464146023
az0.08256880733944955
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I’m sorry, I didn't catch any of that. Do you speak English? I need to know if you witnessed anything unusual during the festival."
detectedLang"en"
heavyLang"en"
scores
en0.8108448928121058
tl0.6115666810530859
no0.4803695150115473
sv0.441340782122905
da0.43785134291068084
hr0.4186046511627907
cs0.41329856584093866
ms0.3902439024390244
it0.3856655290102389
sq0.38313913639479097
pt0.3723849372384937
yo0.3670886075949367
nl0.3630573248407643
ca0.3464052287581699
pl0.34162399414776884
sk0.3353028064992614
et0.3227990970654628
sl0.32228915662650603
fr0.32075471698113206
lt0.30394431554524365
de0.29797191887675506
ro0.23728813559322032
es0.22413793103448276
lv0.21602787456445993
az0.16743755781683625
vi0.16201117318435754
is0.14448669201520914
fi0.14285714285714285
hu0.12959381044487428
be0.07216494845360825
tr0.07216494845360825
uk0.06542056074766354
reliabletrue
passestrue
3
name"Selvi"
dialogue"Mujhe maaf kijiye, mujhe angrezi nahi aati. Par maine wahan... wahan ek parchhayi dekhi. Camera mein... dekhiye!"
detectedLang"fr"
heavyLang"vi"
scores
fr0.46380697050938335
ro0.45701357466063347
de0.4129158512720157
ms0.3442622950819672
tl0.3220338983050847
fi0.29494712103407755
lv0.28825622775800713
en0.28486293206197855
it0.2797118847539016
da0.23175416133162616
ca0.2277992277992278
nl0.22680412371134018
sv0.22580645161290325
es0.2248062015503876
eu0.21568627450980393
et0.21363040629095675
tr0.15730337078651685
pt0.1366906474820144
no0.13544668587896252
reliablefalse
passesfalse
4
name"Hunter"
dialogue"A shadow? You saw a shadow? Hold on, let me see that... wait, is that a figure by the treeline in this shot?"
detectedLang"en"
heavyLang"en"
scores
en0.8137953367875649
tl0.5696107784431138
et0.5670180722891567
nl0.5535714285714286
da0.5240066225165563
fr0.5127118644067797
cs0.5051635111876076
ms0.47963800904977383
yo0.46955719557195574
sl0.4631185807656396
ro0.46261682242990654
it0.4570349386213409
no0.4465832531280077
sk0.4126659856996936
pl0.40476190476190477
sq0.3947368421052632
fi0.3843683083511777
pt0.3823845327604726
de0.3817204301075269
ca0.35174746335963925
hu0.3502824858757062
es0.3451025056947608
hr0.3105515587529976
sv0.30133657351154314
tr0.2830423940149626
is0.22506738544474394
az0.21124828532235942
hi0.1274658573596358
lt0.1248097412480974
eu0.10852713178294573
lv0.10852713178294573
vi0.10852713178294573
ja0.09305993690851734
reliabletrue
passestrue
5
name"Selvi"
dialogue"Haan! Wahi! Woh bahut tezi se chala gaya."
detectedLang"sq"
heavyLang"de"
scores
sq0.4646680942184154
cs0.4600431965442765
fr0.4553376906318083
tl0.44320712694877507
hr0.34725848563968675
sl0.34554973821989526
es0.34210526315789475
en0.32432432432432434
ro0.32432432432432434
pt0.32249322493224936
yo0.299719887955182
fi0.28977272727272724
ms0.2857142857142857
it0.2795389048991354
ca0.2774566473988439
da0.2690058479532163
eu0.258160237388724
sv0.25595238095238093
sk0.23547400611620795
no0.23076923076923078
reliablefalse
passesfalse
6
name"Hunter"
dialogue"Okay, okay... let me try to get this through the translator. So, you saw a figure moving fast? Did you get a clear look at a face?"
detectedLang"en"
heavyLang"en"
scores
en0.8174706649282919
tl0.6321597477666842
no0.5815899581589958
da0.5286195286195287
sl0.4909090909090909
fr0.46605644546147984
pt0.46112394149345654
es0.4492525570416995
nl0.4470774091627172
ro0.44400317712470216
sq0.42716857610474634
cs0.42244224422442245
yo0.41126997476871324
de0.40878378378378377
ms0.388646288209607
sk0.381078691423519
it0.37555753791257807
pl0.35004642525533886
ca0.34148635936030103
sv0.33523266856600187
hr0.3346007604562738
et0.2842535787321063
lv0.27007299270072993
vi0.26854754440961337
lt0.2654774396642183
is0.2
hu0.1822429906542056
tr0.16666666666666666
fi0.16467780429594273
eu0.16367980884109917
az0.15560916767189384
reliabletrue
passestrue
7
name"Selvi"
dialogue"Koshish ki, par andhera bahut tha. Main bahut ghabraayi hui hoon."
detectedLang"fr"
heavyLang"es"
scores
fr0.473314606741573
sq0.3718592964824121
az0.2937853107344633
lv0.29245283018867924
sl0.28977272727272724
eu0.25
hu0.24395161290322584
yo0.23155737704918034
tr0.21711899791231734
en0.20886075949367086
da0.19871794871794873
sv0.19871794871794873
de0.17218543046357618
reliablefalse
passesfalse
8
name"Hunter"
dialogue"I understand. Just keep those files safe. I'll need to bring a forensics team down here to—"
detectedLang"en"
heavyLang"en"
scores
en0.8097915977505789
da0.6328224776500639
no0.612272420768712
sv0.5721726190476191
tl0.528688524590164
de0.5038826574633305
ca0.4739249771271729
nl0.46955719557195574
sq0.4481765834932822
cs0.4401168451801363
pl0.435721295387635
hr0.43405511811023617
sk0.42384769539078154
fr0.40352697095435686
it0.3997912317327766
es0.39088983050847464
et0.3895966029723991
ro0.3876464323748669
yo0.37363834422657954
lv0.3660418963616317
is0.3458475540386803
pt0.3235294117647059
ms0.31710213776722085
lt0.3021844660194175
hu0.27944862155388467
sl0.2684478371501272
az0.25227568270481143
tr0.203601108033241
vi0.18785310734463279
be0.10852713178294573
eu0.10852713178294573
fi0.0987460815047022
uk0.0987460815047022
reliabletrue
passestrue
9
name"Hunter"
dialogue"Hang on, I have to take this. It's the station. Stay right here."
detectedLang"en"
heavyLang"en"
scores
en0.8350197976242851
tl0.7102009273570324
da0.6600181323662738
yo0.629080118694362
no0.6073298429319371
fr0.5577830188679246
pl0.5210727969348659
hr0.5198463508322664
sv0.5142487046632124
sq0.5026525198938991
cs0.4993324432576769
sk0.47405329593267886
et0.423963133640553
de0.41222570532915365
sl0.4113029827315541
lv0.3980738362760835
lt0.39222042139384117
ca0.3872549019607844
fi0.3822075782537067
nl0.3811881188118812
it0.3676222596964587
ro0.3081180811808118
ms0.267578125
tr0.25595238095238093
az0.251497005988024
hi0.1830065359477124
zh0.17035398230088497
be0.15730337078651685
hu0.1515837104072398
is0.14383561643835618
uk0.14383561643835618
vi0.14383561643835618
reliabletrue
passestrue
10
name"Selvi"
dialogue"Kya hua? Aap ja rahe hain? Mujhe madad chahiye!"
detectedLang"et"
heavyLang"fi"
scores
et0.469964664310954
sv0.375
fi0.33774834437086093
eu0.3150684931506849
tl0.3071593533487298
lv0.27710843373493976
ca0.2665036674816626
de0.22680412371134018
sk0.21875
da0.21052631578947367
pl0.21052631578947367
sl0.21052631578947367
sq0.2
yo0.2
reliablefalse
passesfalse
11
name"Hunter"
dialogue"I'll be back, I promise. Don't go anywhere."
detectedLang"en"
heavyLang"en"
scores
en0.7838616714697406
tl0.5804195804195804
pl0.49748743718592964
yo0.47735191637630664
sq0.4746059544658494
cs0.4594594594594595
ro0.441340782122905
hr0.42528735632183906
ca0.40594059405940597
et0.3963782696177062
hu0.38016528925619836
it0.3775933609958506
lt0.3684210526315789
no0.3258426966292135
da0.31662870159453305
sv0.3150684931506849
de0.3135011441647597
pt0.2
sk0.1935483870967742
be0.1891891891891892
es0.1891891891891892
sl0.1891891891891892
tr0.1891891891891892
fr0.18256130790190736
az0.17355371900826447
is0.17355371900826447
lv0.17355371900826447
uk0.17355371900826447
vi0.17355371900826447
ms0.16434540389972147
nl0.16434540389972147
reliabletrue
passestrue