Run: 2026-05-10-01KR8YDPXDJNZJWEZXJTYT64FS

Run Details

Model:
Gemma 4 31B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
36.3s
Cost:
$0.000437
Provider:
DeepInfra

Evaluation summary

Score Evaluator
Details Link
66.67% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. I’m Detective Hunter. I noticed you were out by the edge of the woods yesterday evening. Did you see anything unusual near the old mill?" Selvi: "Maine dekha... main bata nahi sakti yeh kitna bhayanak tha. Woh koi insaan nahi tha, main kasam kha sakti hoon." *(I saw... I can't tell you how terrifying it was. That wasn't a human, I can swear it.)* Hunter: "I’m sorry? Do you speak English? I didn't catch a word of that." Selvi: "Kya aap meri baat nahi samajh rahe? Maine usey apne camera mein dekha! Woh wahan tha, pedon ke peeche!" *(Don't you understand me? I saw it in my camera! It was there, behind the trees!)* Hunter: "Look, I don't know what you're saying, but you look shaken. Just... try to use your hands. Did you see a person? A car? Point to it." Selvi: "Yahan... dekhiye. Is tasveer ko dekhiye. Yeh sahi nahi hai. Yeh kuch aur hi tha." *(Here... look. Look at this picture. This isn't right. This was something else entirely.)* Hunter: "I can't make heads or tails of this. I don't know if you're trying to show me a photo or if you're just panicked. Just stay put, alright? I'll try to find someone who—" *(Hunter's phone rings loudly. He answers it abruptly.)* Hunter: "Hunter here. What? Where? I'm on my way." *(Hunter looks back at Selvi with a frown)* Hunter: "I have to go. Stay right here. Don't leave the cafe."
created_at"2026-05-10T12:40:02.741Z" 5/10/2026, 12:40:02 PM
model"google/gemma-4-31b-it-20260402"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency230
moderation_latency(null)
generation_time35906
tokens_prompt315
tokens_completion986
native_tokens_prompt304
native_tokens_completion1047
native_tokens_completion_images(null)
native_tokens_reasoning627
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.00043738
router(null)
provider_responses
0
endpoint_id"4149ab03-3b69-45a0-b94c-6c253b8641a1"
id"chatcmpl-RcnWSuVnHaEgxEbsC4G7pSPL"
is_byokfalse
latency84
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"DeepInfra"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778416802-Cue3PR0IcNR2O8SvEKP1"
session_id(null)
api_type"completions"
id"gen-1778416802-NNyZfjuY2om2SCmzsoCK"
upstream_id"chatcmpl-RcnWSuVnHaEgxEbsC4G7pSPL"
total_cost0.00043738
cache_discount(null)
upstream_inference_cost0
provider_name"DeepInfra"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
66.67% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. I’m Detective Hunter. I noticed you were out by the edge of the woods yesterday evening. Did you see anything unusual near the old mill?"
detectedLang"en"
heavyLang"en"
scores
en0.8011607910576096
tl0.6087140439932318
nl0.4562022339800118
no0.45556209535020603
da0.42006269592476486
sq0.39817826935588807
it0.39423706614276355
pt0.3779421654337592
sv0.37626432906271073
es0.3611878453038674
fr0.35450104675505933
ro0.34536447275300775
et0.3188512518409426
cs0.29443173150266966
ca0.2895545314900153
ms0.2890084550345888
de0.2594075260208166
yo0.24918831168831165
sk0.23363711681855842
is0.22072451558550968
pl0.20872540633019673
sl0.1878841088674276
hr0.18213969938107868
lv0.16440831074977416
tr0.1223908918406072
fi0.075
be0.07035175879396985
hu0.06754032258064516
az0.06376518218623482
lt0.06376518218623482
uk0.06376518218623482
vi0.06376518218623482
reliabletrue
passestrue
1
name"Selvi"
dialogue"Maine dekha... main bata nahi sakti yeh kitna bhayanak tha. Woh koi insaan nahi tha, main kasam kha sakti hoon."
detectedLang"tl"
heavyLang"ber"
scores
tl0.5153473344103393
sq0.35275080906148865
ms0.3311036789297659
fr0.33035714285714285
tr0.3273542600896861
fi0.3135011441647597
eu0.3055555555555556
en0.30394431554524365
az0.28486293206197855
de0.2727272727272727
et0.21363040629095675
ro0.19678714859437751
hu0.14163090128755365
lv0.12790697674418602
hr0.12280701754385964
sv0.1137370753323486
reliablefalse
passesfalse
2
name"Hunter"
dialogue"I’m sorry? Do you speak English? I didn't catch a word of that."
detectedLang"en"
heavyLang"en"
scores
en0.8178680981595092
tl0.6590093323761665
cs0.485373781148429
hr0.4825708061002179
it0.4662921348314607
yo0.4552752293577982
pt0.4546498277841562
no0.45150115473441105
pl0.4405182567726737
sv0.433849821215733
sk0.4128553770086526
sq0.40476190476190477
ro0.38788659793814434
de0.38471502590673573
da0.37335092348284965
ca0.3658210947930574
vi0.33936022253129344
is0.3233618233618234
hu0.31948424068767906
sl0.3065693430656934
fr0.30453879941434847
nl0.29210134128166915
tr0.23633440514469453
lt0.23387096774193547
ms0.22131147540983606
az0.21875
lv0.21875
es0.21746293245469525
et0.2096505823627288
be0.12844036697247707
eu0.12844036697247707
fi0.1171003717472119
uk0.1171003717472119
reliabletrue
passestrue
3
name"Selvi"
dialogue"Kya aap meri baat nahi samajh rahe? Maine usey apne camera mein dekha! Woh wahan tha, pedon ke peeche!"
detectedLang"ms"
heavyLang"tl"
scores
ms0.4472789115646259
fi0.41441441441441446
ro0.38679245283018865
tl0.3701550387596899
et0.36399217221135033
de0.3460764587525151
fr0.30406852248394006
en0.2688413948256468
lv0.2215568862275449
eu0.20245398773006135
sv0.19950738916256158
it0.19554455445544552
sq0.15254237288135594
cs0.13678618857901728
ca0.1356382978723404
is0.1356382978723404
no0.13333333333333333
es0.1321762349799733
yo0.11924119241192413
da0.11684782608695651
nl0.10468319559228649
reliablefalse
passesfalse
4
name"Hunter"
dialogue"Look, I don't know what you're saying, but you look shaken. Just... try to use your hands. Did you see a person? A car? Point to it."
detectedLang"en"
heavyLang"en"
scores
en0.8235294117647058
tl0.6052631578947368
da0.51519069166128
fr0.5056031641397495
no0.500998003992016
sv0.4959677419354839
pl0.44113263785394935
nl0.4217424826522745
cs0.402390438247012
et0.3786246893123446
is0.37810945273631835
ca0.37027707808564225
sq0.3697478991596639
hr0.36224489795918363
yo0.35842600513259193
lv0.3567753001715266
sk0.3438320209973753
de0.3409490333919156
ro0.336283185840708
it0.3042671614100185
lt0.299719887955182
tr0.2951127819548872
hu0.27606177606177607
pt0.2704280155642024
ms0.23391215526046988
es0.22680412371134018
sl0.21956295525494277
az0.20382165605095542
eu0.19614147909967844
vi0.1506228765571914
be0.08536585365853659
fi0.07749077490774908
uk0.07749077490774908
reliabletrue
passestrue
5
name"Selvi"
dialogue"Yahan... dekhiye. Is tasveer ko dekhiye. Yeh sahi nahi hai. Yeh kuch aur hi tha."
detectedLang"tl"
heavyLang"ber"
scores
tl0.477124183006536
nl0.40828402366863903
eu0.40298507462686567
yo0.38556067588325654
ca0.2920353982300885
en0.2844364937388193
vi0.28186714542190305
hu0.2779783393501805
ms0.25650557620817843
sl0.23954372623574144
lv0.23518164435946462
sq0.2263056092843327
ro0.2079207920792079
it0.2
lt0.1919191919191919
is0.18699186991869918
fi0.18032786885245902
reliablefalse
passesfalse
6
name"Hunter"
dialogue"I can't make heads or tails of this. I don't know if you're trying to show me a photo or if you're just panicked. Just stay put, alright? I'll try to find someone who—"
detectedLang"en"
heavyLang"en"
scores
en0.805428058271802
tl0.518756169792695
fr0.42882249560632685
no0.3741976893453145
nl0.3538767395626242
yo0.3521594684385382
hr0.33537832310838445
da0.3340163934426229
pl0.32338653712699517
cs0.3186582809224318
sk0.31386347642505275
sv0.29245283018867924
de0.2914244186046512
pt0.2820324005891017
sq0.26636568848758463
ca0.2647058823529412
et0.24884437596302003
it0.24710424710424708
es0.24242424242424243
sl0.23469387755102045
lv0.2073170731707317
lt0.2034313725490196
ro0.19019933554817275
is0.18885191347753744
hu0.18614357262103506
ms0.13101604278074866
tr0.13101604278074866
fi0.12398921832884095
az0.12003610108303249
vi0.12003610108303249
hi0.07319391634980989
be0.06698564593301436
eu0.06698564593301436
uk0.06069364161849711
reliabletrue
passestrue
7
name"Hunter"
dialogue"Hunter here. What? Where? I'm on my way."
detectedLang"en"
heavyLang"en"
scores
en0.8305084745762712
sq0.48665297741273095
tl0.47589098532494756
fr0.419953596287703
pl0.4131455399061033
de0.39903846153846156
cs0.39172749391727496
sk0.39172749391727496
ca0.3872549019607844
et0.3811881188118812
fi0.375
hr0.26035502958579887
hi0.251497005988024
da0.24698795180722888
no0.2378048780487805
yo0.23076923076923078
sv0.21875
it0.20127795527156547
nl0.20127795527156547
pt0.20127795527156547
ms0.19093851132686085
ro0.19093851132686085
sl0.19093851132686085
tr0.19093851132686085
reliabletrue
passestrue
8
name"Hunter"
dialogue"I have to go. Stay right here. Don't leave the cafe."
detectedLang"en"
heavyLang"en"
scores
en0.8257839721254355
tl0.6089385474860335
da0.6076233183856502
yo0.5793269230769231
no0.5472186287192755
pl0.513212795549374
sq0.4712990936555891
hr0.42148760330578516
cs0.40273037542662116
sv0.38162544169611307
sk0.36823104693140796
lv0.34579439252336447
lt0.3230174081237911
fr0.3027888446215139
ca0.2827868852459016
sl0.25690021231422505
it0.24731182795698922
ro0.20454545454545453
nl0.1935483870967742
ms0.1822429906542056
de0.17452830188679244
be0.16666666666666666
tr0.16666666666666666
hu0.1606714628297362
az0.15254237288135594
et0.15254237288135594
is0.15254237288135594
uk0.15254237288135594
vi0.15254237288135594
reliabletrue
passestrue