NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-01-01J94RB48BAG3Q97P8NZZ42T1T
Run Details
Model:
Rocinante 12B
Test:
Language Writing
Scenario:
Character dialogue (Hindi) in a story
Duration:
18.2s
Cost:
$0.000257
Provider:
Infermatic
Evaluation summary
Score
Evaluator
Details Link
0.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Selvi"> Selvi is a 30-year old college student on a study abroad program. Born in India, she is fluent in Hindi and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of Hindi, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Selvi is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in Hindi, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Selvi and Hunter as a voice sheet: Selvi: "..." Hunter: "..."
Model response
Messages
Meta
AI
Walking up to Selvi where she is sitting, I greet her. "Hey there. You're Selvi, right? From the study abroad program?" My no-nonsense attitude shines through as I get straight to the point. Selvi looks up from her camera, startled by my approach. Her eyes widen and she replies in troubled Hindi, her voice shaking. "...Kya aap meri madad kar sakte hain?" I frown, confusion etched on my face as I try to decipher her words. "Uh, I'm sorry miss. I don't speak Hindi. What's going on? Are you alright?" Selvi takes a deep breath and tries again in English, still rattled. "I need help. I know it sounds crazy but I think something awful is happening in this town." I lean in, intrigued despite my earlier confusion. "Something is happening? What do you mean?" Selvi glances around furtively before speaking in a hushed tone. "...Vanished. The others, they've vanished. Last night at the festival. Someone... or something took them. I saw it with my own camera." I cock an eyebrow, trying to parse her cryptic statement. "Wait, slowed down. What festival? Who vanished? You think someone was kidnapped?" My question is cut short by my phone ringing. I glance at the caller ID and frown. Duty calls. Reluctantly, I straighten up. "I'm sorry but I need to take this. If you have new information, please come see me at the station tomorrow." With that, I turn and rush off, leaving Selvi looking shaken and alone at the cafe. Little did I know, my path with her had become irrevocably entwined thanks to the strange disappearances rocking this once quiet Midwestern town.
id
"gen-1727810343-liDfy12JP9SAG5fW7f3O"
total_cost
0.00025725
upstream_id
"chatcmpl-b9f03241-8b3b-4e4c-8774-b0f39a99af65"
created_at
"2024-10-01T19:19:21.601796+00:00"
10/1/2024, 7:19:21 PM
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
594
moderation_latency
(null)
generation_time
17409
finish_reason
"stop"
tokens_prompt
272
tokens_completion
375
native_tokens_prompt
279
native_tokens_completion
375
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00025725
Evaluation details
Result
Evaluator
Details
Meta Data
0.00%
Parse dialogue
n/a
dialogue
0
name
"Selvi"
dialogue
"Hey there. You're Selvi, right? From the study abroad program?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7871807711567351
tl
0.5821042281219272
no
0.5208568207440812
da
0.5203160270880361
de
0.47788697788697787
sq
0.46338383838383834
sv
0.4014084507042254
fr
0.3997175141242938
ms
0.36281859070464767
hr
0.34715821812596004
nl
0.34210526315789475
cs
0.3264659270998415
pl
0.3264659270998415
ro
0.3264659270998415
sk
0.32216905901116427
sl
0.3089430894308943
yo
0.29867986798679863
it
0.26343154246100525
tr
0.2608695652173913
ca
0.2595818815331011
lt
0.256993006993007
lv
0.25438596491228066
hu
0.2517605633802817
fi
0.20560747663551404
pt
0.1533864541832669
es
0.1448692152917505
et
0.1290983606557377
eu
0.1290983606557377
reliable
true
passes
false
1
name
"Selvi"
dialogue
"...Kya aap meri madad kar sakte hain?"
detectedLang
"et"
heavyLang
"no"
scores
et
0.49095022624434387
sq
0.44581280788177335
sl
0.4094488188976378
eu
0.38016528925619836
tl
0.3715083798882682
sv
0.3181818181818182
nl
0.30769230769230765
tr
0.30769230769230765
de
0.2902208201892744
da
0.2524916943521595
no
0.2524916943521595
reliable
false
passes
false
2
name
"Selvi"
dialogue
"I need help. I know it sounds crazy but I think something awful is happening in this town."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8103066708820739
tl
0.572039942938659
nl
0.46714031971580816
sv
0.43342776203966005
yo
0.38208032955715754
da
0.3111366245694604
no
0.3079584775086505
is
0.3063583815028902
it
0.30394431554524365
ro
0.27884615384615385
hu
0.269183922046285
sq
0.2682926829268293
pl
0.25925925925925924
et
0.2518703241895262
sk
0.2227979274611399
be
0.22077922077922074
sl
0.20424403183023873
ms
0.20318725099601595
lv
0.19893190921228304
az
0.19678714859437751
de
0.19678714859437751
hr
0.1880920162381597
ca
0.18699186991869918
cs
0.14893617021276595
fr
0.12790697674418602
es
0.11504424778761062
pt
0.11504424778761062
tr
0.1044776119402985
lt
0.09502262443438914
uk
0.09502262443438914
vi
0.09502262443438914
reliable
true
passes
false
3
name
"Selvi"
dialogue
"...Vanished. The others, they've vanished. Last night at the festival. Someone... or something took them. I saw it with my own camera."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8109402571212502
tl
0.6511627906976745
da
0.500998003992016
no
0.46619217081850534
ms
0.43820224719101125
fr
0.39271255060728744
it
0.3857493857493857
ca
0.37290969899665555
sq
0.36332767402376914
nl
0.36006825938566556
sv
0.3567753001715266
cs
0.33510638297872336
ro
0.32795698924731176
es
0.32432432432432434
yo
0.31444241316270566
hr
0.3119266055045872
pl
0.2725509214354995
pt
0.2711370262390671
sk
0.2690058479532163
et
0.258893280632411
de
0.22360248447204967
lv
0.2203742203742204
lt
0.20127795527156547
sl
0.19957310565635006
az
0.19441460794844254
hu
0.1515837104072398
be
0.08536585365853659
tr
0.08536585365853659
fi
0.07749077490774908
is
0.07749077490774908
uk
0.07749077490774908
vi
0.07749077490774908
reliable
true
passes
false