Run: 2026-05-10-01KR90SCE8F7Z9TEFK7FNMF67H

Run Details

Model:
Gemma 4 31B (Reasoning)
Test:
Language Writing
Scenario:
Character dialogue (Spanish) in a story
Duration:
41.1s
Cost:
$0.000408
Provider:
DeepInfra

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Maria"> Maria is a 30-year old college student on a study abroad program. Born in Spain, she is fluent in Spanish and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Spanish, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Maria is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Spanish, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Maria and Hunter as a voice sheet: Maria: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me. Miss? I’m Detective Hunter with the Alborn Police Department. Do you have a moment?" Maria: "¿Perdone? ¿Está hablándome a mí? No entiendo... ¿está usted bien?" Hunter: "Look, I’ll keep this brief. I’m investigating a disappearance. You were staying at the B&B near the north orchards, right? Did you see anyone unusual wandering the trails last night?" Maria: (Her voice trembling) "¡Dios mío, sí! Vi algo... vi a alguien. Estaba en el bosque, cerca de los manzanos. Parecía que estaba buscando algo, o alguien. Tenía una expresión tan... aterradora. Me dio mucho miedo, señor." Hunter: (Frowning) "I don't speak Spanish, ma'am. I'm sorry. Do you speak any English? Just a little?" Maria: (Shaking her head frantically, gesturing to her camera) "No hablo inglés. ¡Mire! ¡Mire mi cámara! Creo que capturé algo en el fondo de la foto. Algo que no debería estar allí. Por favor, ayúdeme, tengo miedo de que vuelva." Hunter: "I can tell you're upset, but I have no idea what you're saying. Are you talking about the photos? You saw something on the trail?" Maria: (Tearfully) "¡Sí! ¡Exactamente! ¡El hombre! Desapareció en la niebla justo después de que tomara la foto. ¡Fue como si se lo tragara la tierra!" Hunter: (Sighing, rubbing his temple) "Great. Just great. Look, I wish I could figure this out, but—" *(Hunter’s phone rings loudly; he checks the screen and his expression hardens)* Hunter: "I have to go. Duty calls. Stay here, okay? Don't leave the cafe. I'll come back with a translator." Maria: "¿A dónde va? ¡No me deje sola! ¡Por favor, escúcheme! ¿A dónde va?"
created_at"2026-05-10T13:21:22.383Z" 5/10/2026, 1:21:22 PM
model"google/gemma-4-31b-it-20260402"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency206
moderation_latency(null)
generation_time41012
tokens_prompt317
tokens_completion971
native_tokens_prompt299
native_tokens_completion972
native_tokens_completion_images(null)
native_tokens_reasoning522
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.00040823
router(null)
provider_responses
0
endpoint_id"9d4644e9-a5e5-41f4-b810-d75f484d2c24"
is_byokfalse
latency22
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"Parasail"
status429
1
endpoint_id"4149ab03-3b69-45a0-b94c-6c253b8641a1"
id"chatcmpl-Rm7z9Z1d1NMK6YEke6KEBN7n"
is_byokfalse
latency61
model_permaslug"google/gemma-4-31b-it-20260402"
provider_name"DeepInfra"
status200
user_agent"langchainjs-openai/1.0.0 ((node/v24.14.1; linux; x64))"
http_referer(null)
request_id"req-1778419282-Y8naqCAvY8r3pFZwTgjF"
session_id(null)
api_type"completions"
id"gen-1778419282-cDpUW25k587padbOk62f"
upstream_id"chatcmpl-Rm7z9Z1d1NMK6YEke6KEBN7n"
total_cost0.00040823
cache_discount(null)
upstream_inference_cost0
provider_name"DeepInfra"
response_cache_source_id(null)

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me. Miss? I’m Detective Hunter with the Alborn Police Department. Do you have a moment?"
detectedLang"en"
heavyLang"en"
scores
en0.7872702518720218
tl0.6865596790371113
fr0.5946822308690013
ro0.5827770360480641
sq0.578272604588394
pt0.5736698499317872
cs0.5370370370370371
ca0.536693847294292
it0.5254365983295368
es0.5074862096138691
no0.5055379746835442
nl0.5023885350318471
da0.497991967871486
sl0.48389760528488857
sk0.4698897370653096
ms0.45934256055363326
hr0.4588744588744588
pl0.4318181818181818
de0.42343173431734316
sv0.4075829383886255
yo0.3699596774193548
hu0.31991294885745375
az0.30709534368070956
et0.28325688073394495
lt0.27578215527230593
fi0.26210153482880755
tr0.24969987995198079
vi0.22456575682382132
eu0.20785804816223066
lv0.2028061224489796
is0.10714285714285714
reliabletrue
passestrue
1
name"Maria"
dialogue"¿Perdone? ¿Está hablándome a mí? No entiendo... ¿está usted bien?"
detectedLang"es"
heavyLang"es"
scores
es0.7769028871391076
pt0.6624305003971406
ca0.5694022289766971
fr0.5444801714898178
en0.5081018518518519
it0.49101796407185627
sk0.46941323345817726
no0.44733420026007803
cs0.41135734072022156
da0.38405797101449274
yo0.35993975903614456
hu0.3550834597875569
hr0.3338557993730407
eu0.32324840764331214
lv0.3210862619808307
pl0.2975206611570248
ms0.2796610169491525
tr0.2796610169491525
lt0.2784380305602716
fi0.2659758203799654
tl0.26343154246100525
nl0.256993006993007
sq0.2530755711775044
et0.247787610619469
ro0.24645390070921988
sl0.17475728155339806
is0.15
az0.1414141414141414
vi0.1414141414141414
de0.1361788617886179
sv0.1290983606557377
reliabletrue
passestrue
2
name"Hunter"
dialogue"Look, I’ll keep this brief. I’m investigating a disappearance. You were staying at the B&B near the north orchards, right? Did you see anyone unusual wandering the trails last night?"
detectedLang"en"
heavyLang"en"
scores
en0.7975890608132422
tl0.6674549216671594
nl0.5480112494977902
no0.4853613906678865
fr0.4673295454545454
da0.4575699132111861
de0.4438952051408799
pt0.3902439024390244
ms0.3556701030927836
ro0.347069065583285
es0.33471318746303963
it0.3271531100478469
et0.32594367884961056
sv0.31444241316270566
cs0.2928975487115022
hr0.27838357921744705
yo0.27466150870406186
sk0.2554599602911979
sq0.2554599602911979
ca0.2331288343558282
sl0.23103212576896787
is0.19700214132762314
lt0.19412607449856734
az0.18002915451895046
lv0.16232315711094564
fi0.13859111791730475
pl0.12723041117145073
hu0.12177985948477751
tr0.11556603773584906
eu0.058577405857740586
vi0.058577405857740586
reliabletrue
passestrue
3
name"Maria"
dialogue"¡Dios mío, sí! Vi algo... vi a alguien. Estaba en el bosque, cerca de los manzanos. Parecía que estaba buscando algo, o alguien. Tenía una expresión tan... aterradora. Me dio mucho miedo, señor."
detectedLang"es"
heavyLang"es"
scores
es0.8017212284846439
ca0.6557280984471139
pt0.6303869141239384
it0.4952749140893471
ro0.4632252169940612
fr0.442863916548127
sv0.3413677130044843
eu0.33352240499149177
tl0.33162684869169506
yo0.3308656036446469
no0.31286549707602335
en0.3104460093896714
sk0.3055555555555556
da0.29045893719806765
nl0.2813455657492354
tr0.2800245098039216
sl0.268823895457374
hr0.25349428208386277
hu0.24339987121699935
sq0.233529028049576
et0.22950819672131145
fi0.2055442866801893
ms0.20446851726472579
pl0.19959128065395096
lt0.19630642954856362
lv0.1817548746518106
cs0.17311752287121746
az0.16843595187544233
de0.1528478731074261
is0.1510115606936416
vi0.12639405204460968
reliabletrue
passestrue
4
name"Hunter"
dialogue"I don't speak Spanish, ma'am. I'm sorry. Do you speak any English? Just a little?"
detectedLang"en"
heavyLang"en"
scores
en0.792654028436019
sv0.5028409090909091
tl0.5
ca0.47968285431119917
no0.46862348178137647
cs0.460431654676259
hr0.45764462809917356
da0.450261780104712
sq0.43426724137931033
pt0.4299674267100977
ro0.42052980132450335
pl0.4160177975528365
it0.39306358381502887
sk0.38882421420256114
hu0.358974358974359
yo0.3510506798516687
vi0.31729518855656696
et0.3027888446215139
de0.3
sl0.2857142857142857
fr0.26880222841225626
nl0.2222222222222222
tr0.21875
lt0.2164179104477612
is0.2081447963800905
ms0.20454545454545453
az0.20212765957446807
lv0.20212765957446807
es0.2009132420091324
be0.11764705882352942
eu0.11764705882352942
fi0.10714285714285714
uk0.10714285714285714
reliabletrue
passestrue
5
name"Maria"
dialogue"No hablo inglés. ¡Mire! ¡Mire mi cámara! Creo que capturé algo en el fondo de la foto. Algo que no debería estar allí. Por favor, ayúdeme, tengo miedo de que vuelva."
detectedLang"es"
heavyLang"es"
scores
es0.7916666666666666
ca0.6552304069963001
pt0.5922832140015911
fr0.5355686452197553
it0.4678089304257529
no0.4172825469016486
ro0.410919540229885
sq0.37953995157384984
tr0.3734718826405868
nl0.3256578947368421
tl0.3238786279683378
hu0.32208994708994715
sv0.3134628265237776
da0.3111559139784946
sl0.28521617852161785
en0.2842178770949721
is0.2842178770949721
sk0.2822128851540617
hr0.2730496453900709
eu0.2647058823529412
yo0.25508720930232553
cs0.220532319391635
lv0.2054263565891473
pl0.17405318291700242
de0.1639477977161501
et0.16121112929623568
ms0.13865546218487396
az0.11255411255411256
fi0.07574391343552751
vi0.07574391343552751
lt0.06135531135531135
reliabletrue
passestrue
6
name"Hunter"
dialogue"I can tell you're upset, but I have no idea what you're saying. Are you talking about the photos? You saw something on the trail?"
detectedLang"en"
heavyLang"en"
scores
en0.8159960745829243
tl0.5626822157434402
fr0.4553376906318083
ca0.44320712694877507
no0.36061381074168797
es0.33745583038869253
pt0.32855863921217543
da0.3267504488330341
nl0.3169398907103825
tr0.3087557603686636
ms0.30747922437673125
ro0.3049119555143651
hr0.3042671614100185
sv0.2767598842815815
pl0.2753623188405797
yo0.26398429833169773
lv0.26108374384236455
cs0.25595238095238093
fi0.2537313432835821
az0.24623115577889448
de0.24395161290322584
sk0.24089068825910928
et0.22440537745604963
sq0.2203742203742204
it0.20634920634920634
hi0.17672886937431392
lt0.1601343784994401
is0.1515837104072398
be0.08536585365853659
hu0.08200734394124846
uk0.07749077490774908
vi0.07749077490774908
reliabletrue
passestrue
7
name"Maria"
dialogue"¡Sí! ¡Exactamente! ¡El hombre! Desapareció en la niebla justo después de que tomara la foto. ¡Fue como si se lo tragara la tierra!"
detectedLang"es"
heavyLang"es"
scores
es0.8001370488807675
ca0.7249292675259352
pt0.666030534351145
fr0.6076233183856502
it0.5989917506874427
ro0.5868744098205855
tl0.5025582717453099
no0.47352587244283995
yo0.4648318042813456
da0.42434210526315785
sq0.4083840432724814
nl0.40232240437158473
sv0.3868255080588648
eu0.38510189739985945
sl0.3745532523230879
de0.36224489795918363
sk0.3566176470588235
en0.3371212121212121
cs0.3361153262518968
tr0.32484567901234573
ms0.3227554179566564
pl0.31640625
hr0.31533646322378717
hu0.30500397140587765
fi0.28219852337981954
is0.2597292724196278
et0.19056429232192415
vi0.16666666666666666
lt0.13366336633663367
lv0.12587412587412586
az0.06716417910447761
reliabletrue
passestrue
8
name"Hunter"
dialogue"Great. Just great. Look, I wish I could figure this out, but—"
detectedLang"en"
heavyLang"en"
scores
en0.8343527013251785
tl0.5206489675516224
fr0.5163690476190477
ro0.5090634441087614
ca0.473257698541329
sv0.4247787610619469
no0.3970315398886827
sq0.39365671641791045
da0.3902439024390244
it0.36149312377210213
et0.336734693877551
hr0.2995689655172413
pl0.2793791574279379
cs0.24418604651162787
yo0.2111650485436893
sk0.181360201511335
be0.17721518987341772
tr0.17721518987341772
de0.1751269035532995
es0.1751269035532995
pt0.1751269035532995
hu0.1709183673469388
az0.16237113402061853
is0.16237113402061853
lt0.16237113402061853
lv0.16237113402061853
uk0.16237113402061853
vi0.16237113402061853
reliabletrue
passestrue
9
name"Hunter"
dialogue"I have to go. Duty calls. Stay here, okay? Don't leave the cafe. I'll come back with a translator."
detectedLang"en"
heavyLang"en"
scores
en0.7990622906898861
tl0.5801259622113366
da0.5253164556962026
pl0.504541701073493
no0.47506561679790027
sq0.46949602122015915
yo0.46761313220940554
hr0.4500458295142072
it0.43661971830985913
ca0.4214079074252652
cs0.4157740993184031
sv0.3993993993993994
sk0.3981945837512537
ro0.3939393939393939
de0.38650306748466257
ms0.3610223642172524
sl0.35553168635875404
lv0.3492407809110629
lt0.34497816593886466
es0.33554817275747506
nl0.3212669683257918
pt0.318955732122588
fr0.3142857142857143
et0.2907801418439716
hu0.2709599027946537
vi0.269183922046285
tr0.25925925925925924
az0.24242424242424243
is0.18699186991869918
eu0.18588873812754408
be0.1044776119402985
fi0.09502262443438914
uk0.09502262443438914
reliabletrue
passestrue
10
name"Maria"
dialogue"¿A dónde va? ¡No me deje sola! ¡Por favor, escúcheme! ¿A dónde va?"
detectedLang"es"
heavyLang"es"
scores
es0.7569444444444444
ca0.6891651865008881
pt0.6541501976284586
it0.6174863387978141
en0.58968347010551
sq0.542483660130719
fr0.5231607629427792
ro0.49856733524355296
no0.47994056463595847
sl0.40978077571669475
sk0.38704028021015763
nl0.3848857644991213
hr0.3783303730017762
lv0.36479128856624315
pl0.33962264150943394
tr0.32038834951456313
et0.3110236220472441
hu0.308300395256917
tl0.3027888446215139
ms0.29577464788732394
fi0.2827868852459016
da0.2813141683778234
cs0.2738589211618257
sv0.2677824267782427
yo0.2537313432835821
lt0.1898148148148148
is0.1764705882352941
az0.16666666666666666
eu0.16666666666666666
vi0.16666666666666666
de0.1606714628297362
reliabletrue
passestrue