NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-01-01J94RHMASYS7EGPNAKJMN3161
Run Details
Model:
Rocinante 12B
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
47.6s
Cost:
$0.000520
Provider:
Infermatic
Evaluation summary
Score
Evaluator
Details Link
66.67%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
"Excuse-moi..." A soft French voice breaks through the clatter of the cafe. Hunter turns to see a young woman with wavy chestnut hair, a camera slung around her neck, hunched over a paper map. He frowns, sizing her up. Early 30s, judging by the fine lines just beginning to appear at the corners of her eyes. Definitely not a local, with her pressed linen dress and designer sandals. Probably one of those traveling VIPs the rich park their kids in for the "authentic" small-town experience. Hunter rolls his eyes. Just what he needs right now. Still, he allows his expression to gentlen, "Yes, I can help. What's the problem?" He asks, pulling out a seat at her table uninvited. The woman startles. "Oh! I am sorry, I thought you were - non, it doesn't matter." She shakes her head. Hunter notices her hands are trembling slightly as she tucks her hair behind her ears. "You are not...professeur...professional man?" "Police, actually," Hunter corrects, shifting in his seat. "What's going on? You look shook up." "Ugh, I...it is just, this town..." She gestures vaguely. Hunter sees her eyes darting around the cozy cafe, as if expecting a threat to sweep in. He leans forward, lowering his voice. "Listen, I'm investigating some incidents in town. If you know something, now is the time to say it." The woman inhales sharply. In rapid French, "J'ai vu des choses étranges ces derniers temps. Des exprimences forstesunanimes! Je ne peux pas en dire plus, ils diraient que je suis fou." Hunter arches an eyebrow. "I wish I could help you but I have no idea what you're saying. You'll have to translate for me." The frightened young woman's eyes widen. "You don't speak French?" Hunter scowls. "Madame, I am not here to play cops and robbers. If you want to speak to me, use English. Otherwise..." he stands up from the table, shrugging his broad shoulders. "Wait! Non! Please..." Her hand grabs his wrist, nails digging in desperately. "Look, I am Marie. Je suis étudiante étrangère. I am here to look at your beautiful...how you say? Lieu? I am fascinated by it." She pauses, swallowing hard. "I have seen things these days. Pushy things. Je ne peux pas dire plus. I'm so sorry." She releases him, folding her hands together tightly. Hunter frowns. This girl was clearly rattled, no question about that. Delusions of grandeur at the least, outright insanity at worst. One thing was for certain - a frightened tourist wasn't going to be much help in tracking down the missing people of Alborn. Just then, Hunter's phone buzzes. He glances at it, frowning. Another tip had come in. He stood, brushing the girl's hands away. "I appreciate you coming forward, Marie, I really do, but I'm going to have to head out. Here, let me give you my card." He scribbles a number on the marker board menu. "If you think of anything else, anything at all, feel free to call me." Hunter turns to leave. "Oh, and welcome to Alborn. Watch yourself, 'kay?" Marie sets down the camera with a clunk. The darkscreen saves it from shattering on the harsh tile floor. Her shoulders slump, eyes trailing Hunter's retreating form. She needs to get out of this place. Fast. But then she remembers her duty to the program, to her semblable students... No. She will not abandon this town, not when its desperate cries have called out to her. Not until the answer is discovered, whatever it may be. Sitting up straighter, Marie absently fiddles with the digital camera's settings. Her decision is made. She will stay, and she will keep delving deeper into the mystery of this cursed town, no matter the consequences. No matter the warning, the danger... She's in far over her head now, but Marie is determined to see it through to the end.
id
"gen-1727810556-FwQhoksiOkMtHWh74T1J"
total_cost
0.00052
upstream_id
"chatcmpl-e794749f-9e1e-4e58-9763-fd6c90d46aec"
created_at
"2024-10-01T19:23:23.973737+00:00"
10/1/2024, 7:23:23 PM
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
3820
moderation_latency
(null)
generation_time
43324
finish_reason
"stop"
tokens_prompt
271
tokens_completion
900
native_tokens_prompt
278
native_tokens_completion
901
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00052
Evaluation details
Result
Evaluator
Details
Meta Data
66.67%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"What's going on? You look shook up."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8188405797101449
tl
0.5954692556634305
et
0.3811881188118812
cs
0.3796526054590571
fi
0.375
fr
0.30362116991643456
ca
0.28366762177650434
hr
0.26035502958579887
hi
0.251497005988024
pl
0.251497005988024
nl
0.2378048780487805
sk
0.23076923076923078
yo
0.23076923076923078
reliable
true
passes
true
1
name
"Hunter"
dialogue
"I wish I could help you but I have no idea what you're saying. You'll have to translate for me."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8225233946434333
no
0.5729813664596274
tl
0.5716510903426791
da
0.5266781411359724
hr
0.4731800766283525
ca
0.4358974358974359
it
0.42886812045690553
pl
0.4246861924686192
es
0.42287513116474296
pt
0.416135881104034
lv
0.41050375133976424
cs
0.40411700975081255
fr
0.4028230184581976
sk
0.394273127753304
sq
0.39159292035398224
tr
0.3685419058553387
sl
0.3276283618581907
ro
0.3184634448574969
lt
0.298469387755102
yo
0.289405684754522
ms
0.2857142857142857
et
0.28292046936114734
sv
0.2637215528781794
vi
0.21540656205420827
nl
0.2109038737446198
az
0.18154761904761904
fi
0.12
be
0.11290322580645161
de
0.1085899513776337
eu
0.1085899513776337
hu
0.1085899513776337
is
0.10277324632952692
uk
0.10277324632952692
reliable
true
passes
true
2
name
"Hunter"
dialogue
"Madame, I am not here to play cops and robbers. If you want to speak to me, use English. Otherwise..."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8028909329829172
tl
0.5833333333333334
no
0.4157740993184031
sq
0.3890020366598778
nl
0.3871297242083759
et
0.38461538461538464
hr
0.35275080906148865
yo
0.34994582881906827
da
0.3442622950819672
cs
0.3435448577680525
sv
0.34065934065934067
ro
0.3281075027995521
de
0.318955732122588
pl
0.30313588850174217
ca
0.26380368098159507
sl
0.2583436341161928
sk
0.2537313432835821
it
0.24528301886792453
lv
0.23566878980891717
lt
0.21773142112125163
ms
0.20634920634920634
tr
0.19786096256684493
es
0.1452991452991453
fr
0.13169319826338638
pt
0.12790697674418602
hi
0.12663755458515283
fi
0.1111111111111111
be
0.1044776119402985
hu
0.10044977511244377
az
0.09502262443438914
is
0.09502262443438914
uk
0.09502262443438914
vi
0.09502262443438914
reliable
true
passes
true
3
name
"Marie"
dialogue
"I have seen things these days. Pushy things. Je ne peux pas dire plus. I'm so sorry."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7476022211004543
fr
0.5974235104669887
sq
0.54995499549955
it
0.5112414467253177
sl
0.4747899159663866
da
0.46524064171123
hr
0.4582881906825569
nl
0.44382647385984425
yo
0.4292237442922375
tl
0.41245593419506466
cs
0.4004796163069544
sk
0.3924665856622114
no
0.37888198757763975
ca
0.3757802746566792
de
0.3734335839598998
sv
0.3646759847522236
ro
0.3581514762516046
lt
0.3464052287581699
pl
0.302649930264993
lv
0.2492492492492493
tr
0.24698795180722888
vi
0.24471299093655588
is
0.23195084485407066
hu
0.2295839753466872
pt
0.16666666666666666
fi
0.14965986394557826
ms
0.14383561643835618
be
0.12280701754385964
az
0.11190053285968028
et
0.11190053285968028
uk
0.11190053285968028
reliable
true
passes
false
4
name
"Marie"
dialogue
"If you think of anything else, anything at all, feel free to call me."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8353909465020576
tl
0.6389891696750902
no
0.5698924731182796
sq
0.4444444444444444
nl
0.43661971830985913
da
0.4108983799705449
yo
0.38461538461538464
cs
0.34318555008210183
sl
0.34318555008210183
et
0.313893653516295
sv
0.3006993006993007
ms
0.28825622775800713
de
0.27927927927927926
it
0.2753623188405797
pl
0.2523364485981308
sk
0.24812030075187969
lv
0.23371647509578544
hr
0.2248062015503876
lt
0.20634920634920634
es
0.20318725099601595
fr
0.18533604887983707
is
0.18200408997955012
pt
0.18032786885245902
ro
0.18032786885245902
fi
0.15789473684210525
reliable
true
passes
false
5
name
"Hunter"
dialogue
"Oh, and welcome to Alborn. Watch yourself, 'kay?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7758620689655172
tl
0.4574290484140233
yo
0.4206773618538324
de
0.38679245283018865
pl
0.2934782608695652
sk
0.28884026258205686
cs
0.2872807017543859
nl
0.2777777777777778
lv
0.27293064876957496
sl
0.2713004484304933
hr
0.2630385487528345
no
0.2511520737327189
da
0.24242424242424243
lt
0.24242424242424243
fr
0.21875
sv
0.21875
it
0.1875
reliable
true
passes
true