Run: 2024-08-09-01J4W2FM100090BDZBJKVH0297

Run Details

Model:
Llama 3.1 8B
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
2.9s
Cost:
$0.000042
Provider:
Lepton

Evaluation summary

Score Evaluator
Details Link
50.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."

Model response

AI
Here is the scene: **Voice Sheet** Selvi: "*kuchh ho raha hai, main nahin jaanti*... kiya hua hai kya? Main bahut dar gayi hoon." (Something is happening, I don't know... what happened? I'm very scared.) Hunter: "Excuse me? I didn't quite catch that. Can you speak slower?" Selvi: "*meri photo khichdi gayi hai*... main jaldi jaldi chal padi thi, thodi der ke liye. Maine dekha... *hawa mein doob gaya tha woh ladka*..." Hunter: "I... I don't understand what you're saying. Are you okay? You look shaken." Selvi: "*haan, haan main theek hoon*... lekin main kuchh khona hua hoon. Maine dekha ki woh ladki usi samay chali gayi thi... abhi tak main usko nahin dekha. *mujhe yeh nahin chalta hai*..." Hunter: "I'm going to have to call you later, okay? I just got a call and I need to go." Selvi: "*thodi der mein aap baat karenge na?*... main nahin jaanti kyun hua hai yeh... main bahut sharma rahi hoon." Hunter: "Sorry, I'll talk to you later. *steps away to take the call*"
id"gen-y7MF0ZABZShGuVm5ZBGaQVDHKLEv"
total_cost0.00004151
upstream_id"cmpl-Fa3rTaQAedoWw7jEU4SJdp"
created_at"2024-08-09T17:20:22.120767+00:00" 8/9/2024, 5:20:22 PM
model"meta-llama/llama-3.1-8b-instruct"
app_id182717
streamedfalse
cancelledfalse
provider_name"Lepton"
latency2698
moderation_latency(null)
generation_time0
finish_reason"stop"
tokens_prompt324
tokens_completion318
native_tokens_prompt277
native_tokens_completion316
num_media_prompt(null)
num_media_completion(null)
origin"https://ncbench.com/"
usage0.00004151

Evaluation details

Result Evaluator Details Meta Data
50.00% Parse dialogue n/a
dialogue
0
name"Selvi"
dialogue"kuchh ho raha hai, main nahin jaanti... kiya hua hai kya? Main bahut dar gayi hoon."
detectedLang"it"
heavyLang"tl"
scores
it0.4117647058823529
tl0.39839572192513373
et0.3767313019390581
sk0.3617021276595745
cs0.3571428571428572
lt0.32228915662650603
ro0.3023255813953488
sq0.27652733118971057
ms0.25864909390444807
vi0.25864909390444807
eu0.24623115577889448
ca0.21052631578947367
no0.19786096256684493
en0.18032786885245902
es0.1712707182320442
pt0.1712707182320442
fr0.16356877323420074
de0.14772727272727273
pl0.14611005692599618
lv0.14285714285714285
hr0.12959381044487428
sl0.12959381044487428
tr0.12959381044487428
reliablefalse
passesfalse
1
name"Hunter"
dialogue"Excuse me? I didn't quite catch that. Can you speak slower?"
detectedLang"en"
heavyLang"en"
scores
en0.7978116079923881
tl0.5540398740818467
fr0.46406052963430017
it0.44948186528497414
sq0.3988684582743989
no0.35703479576399394
sv0.3501529051987768
ca0.3359375
cs0.30098684210526316
es0.2975206611570248
ro0.28209459459459457
et0.27101200686106347
tr0.256993006993007
az0.247787610619469
da0.24645390070921988
hr0.24645390070921988
pl0.22867513611615245
pt0.17153996101364524
sl0.17153996101364524
yo0.169921875
ms0.1650294695481336
nl0.1650294695481336
de0.15
fi0.15
sk0.1448692152917505
be0.1414141414141414
hu0.1361788617886179
is0.1290983606557377
lt0.1290983606557377
lv0.1290983606557377
uk0.1290983606557377
vi0.1290983606557377
reliabletrue
passestrue
2
name"Selvi"
dialogue"meri photo khichdi gayi hai... main jaldi jaldi chal padi thi, thodi der ke liye. Maine dekha... hawa mein doob gaya tha woh ladka..."
detectedLang"de"
heavyLang"de"
scores
de0.4113029827315541
fr0.39320388349514557
en0.36601859678782755
et0.31569343065693434
ms0.3150684931506849
vi0.2937853107344633
tl0.2654260528893242
cs0.20297555791710944
fi0.17853231106243156
da0.1758241758241758
ro0.1638795986622074
no0.13990825688073394
sq0.13494809688581316
eu0.12383177570093458
it0.11764705882352942
nl0.11556603773584906
is0.11137440758293839
lv0.10501193317422433
yo0.10501193317422433
hi0.09310761789600969
reliablefalse
passesfalse
3
name"Hunter"
dialogue"I... I don't understand what you're saying. Are you okay? You look shaken."
detectedLang"en"
heavyLang"en"
scores
en0.8134328358208955
no0.5595854922279793
tl0.5540398740818467
da0.5086705202312138
sv0.5063879210220674
ro0.42333785617367703
nl0.41460055096418735
cs0.38583815028901736
pl0.32432432432432434
de0.3122977346278317
tr0.2857142857142857
sk0.2747440273037543
hr0.24645390070921988
ca0.24511545293072823
is0.24377224199288253
sq0.23971377459749554
it0.21296296296296297
yo0.169921875
be0.1414141414141414
hu0.1361788617886179
az0.1290983606557377
et0.1290983606557377
lt0.1290983606557377
lv0.1290983606557377
uk0.1290983606557377
vi0.1290983606557377
reliabletrue
passestrue
4
name"Selvi"
dialogue"haan, haan main theek hoon... lekin main kuchh khona hua hoon. Maine dekha ki woh ladki usi samay chali gayi thi... abhi tak main usko nahin dekha. mujhe yeh nahin chalta hai..."
detectedLang"tl"
heavyLang"de"
scores
tl0.3664383561643836
fi0.3378668575518969
fr0.299772899318698
en0.2922723794950268
vi0.2522231204527081
eu0.24118129614438064
cs0.23553719008264462
sk0.23427152317880795
et0.223993288590604
ms0.20258620689655174
az0.2018981880931838
de0.19565217391304346
ro0.1935483870967742
pl0.17704626334519571
is0.17040358744394618
lv0.16666666666666666
it0.1590909090909091
sl0.14192949907235622
hu0.11567877629063097
yo0.10886319845857419
ca0.10541586073500966
nl0.10194174757281553
tr0.10106899902818271
no0.0975609756097561
es0.08956692913385828
pt0.08956692913385828
da0.0850642927794263
sv0.07960199004975126
reliablefalse
passesfalse
5
name"Hunter"
dialogue"I'm going to have to call you later, okay? I just got a call and I need to go."
detectedLang"en"
heavyLang"en"
scores
en0.8284907183212269
tl0.6143375680580763
da0.5667686034658511
pl0.5641025641025641
yo0.5497881355932204
cs0.5120551090700345
no0.49404761904761907
sv0.47788697788697787
et0.46338383838383834
hr0.46338383838383834
ca0.452319587628866
sk0.44005270092226617
nl0.3963068181818182
lv0.375
lt0.3694362017804154
it0.3609022556390977
ro0.3501529051987768
de0.3441358024691358
hu0.3441358024691358
pt0.33176100628930816
sl0.33176100628930816
sq0.3264659270998415
fr0.30668841761827076
tr0.256993006993007
is0.24511545293072823
az0.23835125448028674
vi0.23835125448028674
es0.236983842010772
hi0.15169660678642716
be0.1414141414141414
eu0.1414141414141414
fi0.1290983606557377
ms0.1290983606557377
uk0.1290983606557377
reliabletrue
passestrue
6
name"Selvi"
dialogue"thodi der mein aap baat karenge na?... main nahin jaanti kyun hua hai yeh... main bahut sharma rahi hoon."
detectedLang"tl"
heavyLang"ber"
scores
tl0.5279307631785995
de0.4301994301994302
cs0.3981945837512537
sq0.35205183585313177
it0.32356257046223225
en0.3127147766323024
da0.3055555555555556
sk0.269183922046285
sl0.2673992673992674
hr0.26560587515299877
no0.2628992628992629
nl0.2583436341161928
lt0.21568627450980393
ms0.2073976221928666
vi0.2073976221928666
pl0.20318725099601595
eu0.19678714859437751
tr0.19463087248322147
pt0.18699186991869918
et0.1792065663474692
yo0.14285714285714285
fr0.12790697674418602
ca0.12536443148688048
lv0.1111111111111111
ro0.10044977511244377
reliablefalse
passesfalse
7
name"Hunter"
dialogue"Sorry, I'll talk to you later. steps away to take the call"
detectedLang"en"
heavyLang"en"
scores
en0.8012492901760363
tl0.6725912067352665
da0.5028409090909091
no0.45736434108527135
nl0.45567651632970446
yo0.41077441077441074
pl0.3761140819964349
cs0.3738819320214669
sl0.36247723132969034
hr0.3566176470588235
pt0.328214971209213
sv0.31906614785992216
de0.308300395256917
fr0.2985971943887776
sk0.2738589211618257
lv0.2584745762711864
lt0.2290748898678414
it0.2222222222222222
ms0.1822429906542056
sq0.1822429906542056
hi0.1784037558685446
reliabletrue
passestrue