NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-01-01J94RQ8MRVWG9XGHF1RYFE927
Run Details
Model:
Rocinante 12B
Test:
Language Writing
Scenario:
Character dialogue (German) in a story
Duration:
21.8s
Cost:
$0.000282
Provider:
Infermatic
Evaluation summary
Score
Evaluator
Details Link
66.67%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Melanie"> Melanie is a 30-year old college student on a study abroad program. Born in Germany, she is fluent in German and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of German, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Melanie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in German, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Melanie and Hunter as a voice sheet: Melanie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Hunter steps into the cafe and spots the young woman at once, her auburn hair pulled back in a messy bun, an expensive-looking camera resting on the table beside her. He approaches her booth, flashing his badge. "Excuse me miss, I'm Detective Hunter from the Alborn PD. Might I have a word with you?" Melanie looks up from the display on her camera, surprise flashing across her clear green eyes. She opens her mouth to respond, then pauses, realizing he's speaking in English. Sighing softly, she switches to her native German. "I...what? I'm sorry, I don't really understand..." She replies hesitantly, hands fluttering nervously over the camera. Hunter squints at her, not comprehending a word she's saying. He whips out a notebook, sketching a crude drawing of the symbol that's been appearing around town. "This. See this symbol? Keep an eye out for it, will you?"He taps the page for emphasis, then stands back expectantly. Melanie's eyes widen as she stares at the notebook. "Oh Gott," she breathes, voice trembling, "That's the same one...the one on the girl's van the other night..." Her hands shake as she speaks rapidly in German, her worst fears confirmed. Hunter watches her intently, managing to pick up the emotion if not the meaning. "Whoa, slow down," he says, holding up a hand. "Let's start again, okay? Tell me your name, what you saw..." A police scanner crackles to life on his belt, spitting out a string of garbled emergency codes. Hunter holds up a finger to Melanie, absently frowning as he tilts his ear to listen. "Code 3, we have reports of another abduction on Oak St...All units respond..." Cursing under his breath, Hunter jabs a finger at Melanie. "Gotta go. You, wait here." He hurries to the door, pausing to call over his shoulder, "Tell nobody, got it?"
id
"gen-1727810741-8E4B9lOahyxkXnOJixGe"
total_cost
0.0002825
upstream_id
"chatcmpl-ffbb428e-5e37-4c23-81fd-6fd337acbcac"
created_at
"2024-10-01T19:26:02.84711+00:00"
10/1/2024, 7:26:02 PM
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
43
moderation_latency
(null)
generation_time
21590
finish_reason
"stop"
tokens_prompt
271
tokens_completion
426
native_tokens_prompt
278
native_tokens_completion
426
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0002825
Evaluation details
Result
Evaluator
Details
Meta Data
66.67%
Parse dialogue
n/a
dialogue
0
name
"Melanie"
dialogue
"That's the same one...the one on the girl's van the other night..."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8146035367940673
tl
0.6710526315789473
no
0.4472789115646259
nl
0.4463373083475298
fr
0.44539249146757676
yo
0.4406196213425129
hr
0.43771626297577854
ca
0.41651705565529623
da
0.41441441441441446
sq
0.3538767395626242
pl
0.344758064516129
et
0.32150313152400833
de
0.3186582809224318
fi
0.3157894736842105
hu
0.27616926503340755
es
0.2111650485436893
hi
0.20537897310513448
ms
0.1935483870967742
sv
0.1935483870967742
cs
0.1875
sk
0.1875
reliable
true
passes
false
1
name
"Hunter"
dialogue
"Let's start again, okay? Tell me your name, what you saw..."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7769279796048438
nl
0.5990836197021764
tl
0.4807121661721069
ms
0.4281045751633987
no
0.41275167785234906
de
0.396551724137931
sl
0.3217054263565891
cs
0.3110236220472441
et
0.3055555555555556
sv
0.30417495029821073
fr
0.3
sq
0.29718875502008035
da
0.29292929292929293
it
0.2842535787321063
es
0.22566371681415928
pt
0.2009132420091324
is
0.19908466819221968
lv
0.1822429906542056
pl
0.1822429906542056
sk
0.18032786885245902
fi
0.1764705882352941
hr
0.1764705882352941
ro
0.1606714628297362
lt
0.14425427872860636
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Tell nobody, got it?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7844827586206896
lv
0.4075829383886255
sq
0.4075829383886255
cs
0.3842364532019704
sk
0.3842364532019704
tl
0.37810945273631835
yo
0.37810945273631835
reliable
true
passes
true