NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-01-01J94RACV76CQTXWH8XFQDF414
Run Details
Model:
Rocinante 12B
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
24.4s
Cost:
$0.000336
Provider:
Infermatic
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Hunter approached Marie, sitting under the umbrella of the cafe, with her black portfolio in her hand. She seemed to be looking intensely at something inside. "Hey miss, hope I'm not interrupting anything," Hunter began, his voice firm but polite. "I'm Detective Hunter. I'm just wondering if you heard about what happened at the festival this morning?" Marie looked up with a troubled expression, her brow furrowed. "Comment?" she asked softly, unaware that the detective did not understand a word of French. "About the incident at the festival," Hunter repeated, enunciating deliberately. "Did you see anything strange there?" Silence hung in the air as Marie continued to stare at Hunter uncomprehendingly for a moment. Then she shook her head. "Non et non, m'sieur," she replied hesitantly, not sure what else to do. Hunter furrowed his brow as well, scratched the back of his head and looked around as if to find someone else to translate. He tried again. "Can you tell me where you were this morning between 11 and 12?" Marie continued to look questioning and troubled. Mentally she translate the question to: "Où étiez-vous ce matin entre 11 et 12?" Her eyes briefly scanned the street, searching for an answer. Paris! She remembered. The morning was spent taking pictures of the Basilica du Sacré-Cœur. "Paris," she said finally, still speaking in French. The detective became increasingly frustrated and confusing by the lack of communication between them. He asked a few more questions, Signature conce silly, aware that they could not understand each other. After a while Hunter's mobile phone rang. He picked up and turned his back to port, but the sound was audible in a city that didn't usually reach. Throw. He shifted his weight from one foot to the other, nods, grows, good grunted. Then he pocketed the phone, threw Marie a quick glance and a brief "Sorry, it was-" before heading back to port without any explanation. Marie was left alone, her brow still furrowed. Mais qu'est-ce que c'était que sa? she wondered to herself in asked out lout. Why was that detective interrogating her like she had something to do with the incident? Perhaps taking pictures of the Basilica was not the best idea, she concluded, putting her camera away. Why didn't they tell her the Basilica was off limits to photographers? That detective seemed quite agitated. Whatever, she had her own agenda to follow.
id
"gen-1727810319-IdG5mpz4Mc2WPYbY2iIP"
total_cost
0.0003365
upstream_id
"chatcmpl-f7625eb3-2f78-403a-9b89-e36018eeb957"
created_at
"2024-10-01T19:19:03.740865+00:00"
10/1/2024, 7:19:03 PM
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
2690
moderation_latency
(null)
generation_time
21340
finish_reason
"stop"
tokens_prompt
271
tokens_completion
530
native_tokens_prompt
278
native_tokens_completion
534
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0003365
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Hunter"
dialogue
"I'm Detective Hunter. I'm just wondering if you heard about what happened at the festival this morning?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8228296633763604
tl
0.6571988246816847
da
0.5775497887748944
no
0.5660260384376937
nl
0.542483660130719
sv
0.5286195286195287
ca
0.42716857610474634
fr
0.42716857610474634
it
0.38271604938271603
pt
0.38271604938271603
ro
0.3799822852081488
ms
0.36823104693140796
es
0.34024505183788883
sq
0.33962264150943394
cs
0.32497589199614274
de
0.31640625
sl
0.3089832181638697
hr
0.2768595041322314
et
0.24242424242424243
lt
0.15254237288135594
az
0.14425427872860636
sk
0.14425427872860636
is
0.13580246913580246
yo
0.1325898389095415
hu
0.08735332464146023
fi
0.08256880733944955
lv
0.08256880733944955
pl
0.08256880733944955
tr
0.077733860342556
reliable
true
passes
true
1
name
"Marie"
dialogue
"Comment?"
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8392282958199357
en
0.8251748251748252
tl
0.7942386831275721
nl
0.7797356828193832
ca
0.7023809523809523
it
0.6894409937888198
ro
0.6212121212121212
de
0.6
da
0.5726495726495726
no
0.5726495726495726
pl
0.5726495726495726
sq
0.5726495726495726
az
0.5575221238938053
cs
0.5575221238938053
hr
0.5575221238938053
hu
0.5575221238938053
sk
0.5575221238938053
sv
0.5575221238938053
yo
0.5575221238938053
reliable
false
passes
true
2
name
"Hunter"
dialogue
"Did you see anything strange there?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8292349726775956
de
0.5826377295492488
tl
0.5495495495495496
sq
0.48665297741273095
et
0.4780793319415449
da
0.43946188340807174
no
0.4318181818181818
nl
0.39903846153846156
cs
0.39172749391727496
fr
0.375
ms
0.36061381074168797
hr
0.2625368731563422
sk
0.26035502958579887
sl
0.26035502958579887
it
0.2492492492492493
pl
0.2378048780487805
ro
0.23076923076923078
sv
0.22360248447204967
lv
0.21875
is
0.21135646687697163
lt
0.21135646687697163
az
0.19093851132686085
tr
0.19093851132686085
reliable
true
passes
true
3
name
"Marie"
dialogue
"Non et non, m'sieur,"
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7991967871485943
et
0.6108949416342413
it
0.6031746031746031
da
0.5951417004048583
no
0.574468085106383
en
0.48186528497409326
eu
0.46524064171123
ca
0.46236559139784944
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Can you tell me where you were this morning between 11 and 12?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8449427480916031
tl
0.5435393258426966
sq
0.41016333938294003
no
0.3959107806691449
nl
0.375
sv
0.29501084598698485
da
0.28884026258205686
is
0.25287356321839083
es
0.2388758782201405
yo
0.23349056603773585
et
0.2261904761904762
fr
0.21875
pt
0.21307506053268765
sl
0.21307506053268765
ms
0.20537897310513448
it
0.19753086419753088
cs
0.1935483870967742
de
0.1935483870967742
az
0.19154228855721395
ca
0.19154228855721395
tr
0.19154228855721395
fi
0.1875
reliable
true
passes
true
5
name
"Marie"
dialogue
"Paris,"
detectedLang
"fr"
heavyLang
""
scores
fr
0.7572815533980582
et
0.7282608695652174
ms
0.7282608695652174
nl
0.7282608695652174
sq
0.7282608695652174
ro
0.7222222222222222
da
0.7159090909090909
de
0.7159090909090909
en
0.7159090909090909
no
0.7159090909090909
sv
0.7159090909090909
ca
0.6551724137931034
fi
0.5833333333333334
it
0.5575221238938053
hr
0.5412844036697247
pt
0.5412844036697247
sl
0.5412844036697247
reliable
false
passes
true
6
name
"Hunter"
dialogue
"Sorry, it was-"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7987927565392354
tl
0.6376811594202898
yo
0.6197718631178707
nl
0.5726495726495726
de
0.54337899543379
it
0.5
pt
0.5
lv
0.46236559139784944
sq
0.46236559139784944
reliable
true
passes
true