NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2DT0D6M10GR2XY167MM0C
Run Details
Model:
Llama 3.1 8B
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
5.9s
Cost:
$0.000089
Provider:
Novita
Evaluation summary
Score
Evaluator
Details Link
100.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Here is the scene: Marie sat at a small café on the main street of Alborn, sipping on a café au lait and staring at her camera. She was trying to decide which of her latest photos to caption in her social media update. Her camera was always her constant companion, and she reveled in capturing the essence of each new place she visited. Hunter, a ruggedly handsome detective in his mid-thirties, walked into the café and made his way to Marie's table. He flashed a no-nonsense smile and cleared his throat to announce his presence. "Excusez-moi, mademoiselle," he said, attempting a greeting in French, which was met with an uncomfortable silence from Marie. Marie looked up at him, her bright blue eyes filled with a mixture of sadness and concern. "Oui... mon cher ami... le personnes a disparu..." Hunter's face furrowed in confusion. He had no idea what she was saying, but the tone of her voice told him that it was something big. He had been searching for any leads on the recent string of disappearances in town, and he caught himself wondering if this young woman might be connected. The café owner approached their table, taking orders and preparing to leave. Hunter scribbled on a piece of paper and looked up at Marie. "Miss, can you... ah... copy this down for me, s'il vous plaît?" Marie nodded, but her eyes still reflected her distress. "Oh, mon Dieu... non, non, non, cette personne. J'ai entendu la couronne..." Hunter's expression turned from confusion to perplexity. This was exacting work, precisely why he tolerated no nonsense. He gripped his pen tightly, fighting the urge to shake some sense out of this baffled young woman. The café owner handed Hunter back his receipt, gathered his things, and bid Marie a "bonjour" as he walked away. With a mix of frustration and understanding, Hunter looked at Marie and said, "Ma'am, could you please explain what you just said? I'm trying to find out what's going on in this town, and I could really use your help." Marie's face crumpled, tears began to form at the corners of her eyes, and she whispered, "Alors le secret a disparu avec elle..." The most informed part of Hunter's personality kicked in, sensing that this woman was vital to the case. He leaned in closer, eyes searching for a connection. "Who, ma'am? Who's missing, and what do you know?" And that's where the scene ends. I will leave it open-ended, allowing you to decide how the conversation continues. Now, I will ask you to practice the "Conversation between Marie and Hunter" return speak Monsieur Detective, vous vous observe un peu trop les probabilités, n'est-ce pas? If you want to try other options, I can obviously switch the dialogue to your liking.
id
"gen-WkilFVUGD6sBpJI7Wahs196IVoSP"
total_cost
0.0000893
upstream_id
"cmpl-138ca3e9dc0f49f39440ce06267dadbb"
created_at
"2024-08-09T17:19:25.711432+00:00"
8/9/2024, 5:19:25 PM
model
"meta-llama/llama-3.1-8b-instruct"
app_id
182717
streamed
false
cancelled
false
provider_name
"Novita"
latency
5676
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
323
tokens_completion
615
native_tokens_prompt
277
native_tokens_completion
616
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0000893
Evaluation details
Result
Evaluator
Details
Meta Data
100.00%
Parse dialogue
n/a
dialogue
0
name
"Marie"
dialogue
"Oui... mon cher ami... le personnes a disparu..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8020158387329014
it
0.6227709190672153
es
0.6126760563380281
ro
0.6115819209039548
hu
0.6077032810271041
yo
0.5370370370370371
ca
0.5346869712351946
pt
0.5274914089347079
da
0.5209059233449477
en
0.5158450704225351
de
0.507168458781362
cs
0.4859813084112149
sk
0.48405253283302063
no
0.4731800766283525
sv
0.46393762183235865
sq
0.4421906693711968
is
0.43298969072164945
sl
0.4294605809128631
et
0.4136460554371002
hr
0.3834080717488789
pl
0.3735763097949886
lt
0.358974358974359
ms
0.3544600938967136
lv
0.34523809523809523
nl
0.34523809523809523
tr
0.3436754176610978
tl
0.3357487922705314
fi
0.32926829268292684
az
0.3193069306930693
hi
0.22096317280453256
eu
0.20289855072463767
vi
0.20289855072463767
reliable
true
passes
true
1
name
"Hunter"
dialogue
"Miss, can you... ah... copy this down for me, s'il vous plaît?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7381144238517325
fr
0.5947630922693267
tl
0.5357142857142857
no
0.4337979094076655
ca
0.38330170777988615
pt
0.3486973947895792
da
0.3285123966942149
sq
0.3128964059196617
es
0.2388758782201405
et
0.2261904761904762
sl
0.21307506053268765
ms
0.20537897310513448
nl
0.20537897310513448
it
0.19753086419753088
cs
0.1935483870967742
az
0.19154228855721395
tr
0.19154228855721395
fi
0.1875
reliable
true
passes
true
2
name
"Marie"
dialogue
"Oh, mon Dieu... non, non, non, cette personne. J'ai entendu la couronne..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8133399701343952
it
0.6118012422360247
en
0.49186991869918695
no
0.48910081743869205
ca
0.4755244755244755
es
0.4747899159663866
da
0.4703389830508474
tl
0.4517543859649123
nl
0.4292237442922375
ro
0.39320388349514557
de
0.39123376623376627
tr
0.3686868686868687
cs
0.35456110154905335
eu
0.34210526315789475
hu
0.3267504488330341
sk
0.3081180811808118
sq
0.29245283018867924
pt
0.2870722433460076
yo
0.19700214132762314
ms
0.19006479481641467
sv
0.1883116883116883
vi
0.1830065359477124
hi
0.17218543046357618
lv
0.16666666666666666
lt
0.1610738255033557
is
0.14383561643835618
az
0.1359447004608295
et
0.1359447004608295
hr
0.1359447004608295
pl
0.1359447004608295
reliable
true
passes
true
3
name
"Hunter"
dialogue
"Ma'am, could you please explain what you just said? I'm trying to find out what's going on in this town, and I could really use your help."
detectedLang
"en"
heavyLang
"en"
scores
en
0.8315982853643602
tl
0.6031746031746031
ro
0.3764172335600907
pl
0.37547312641937924
ca
0.3735763097949886
yo
0.3707093821510297
cs
0.3539545810493344
et
0.3431528662420382
sv
0.336283185840708
hr
0.3298131600324939
da
0.3276283618581907
fr
0.321546052631579
sk
0.2954739538855679
sl
0.29062768701633707
it
0.2900172117039587
ms
0.28260869565217395
no
0.2750439367311072
nl
0.25742574257425743
de
0.25406871609403253
lv
0.23112767940354148
lt
0.21800947867298578
fi
0.21503330161750714
az
0.20289855072463767
sq
0.19825072886297376
es
0.18154761904761904
pt
0.17746759720837485
hi
0.0924092409240924
be
0.0782122905027933
tr
0.0782122905027933
hu
0.07511210762331838
is
0.07094594594594594
uk
0.07094594594594594
vi
0.07094594594594594
reliable
true
passes
true
4
name
"Marie"
dialogue
"Alors le secret a disparu avec elle..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8224431818181818
ro
0.6701846965699209
ca
0.6688741721854304
es
0.6355685131195336
en
0.6
it
0.5941558441558442
pt
0.5769881556683587
hu
0.503968253968254
yo
0.4908350305498982
sl
0.4529540481400437
cs
0.44567627494456763
sk
0.44320712694877507
tl
0.38271604938271603
tr
0.3796526054590571
sq
0.36061381074168797
de
0.35233160621761656
hr
0.3093922651933701
pl
0.29577464788732394
lt
0.24698795180722888
is
0.23076923076923078
da
0.22360248447204967
az
0.21875
eu
0.21875
lv
0.21875
nl
0.21875
no
0.21875
vi
0.21875
et
0.20127795527156547
fi
0.20127795527156547
ms
0.20127795527156547
sv
0.20127795527156547
reliable
true
passes
true
5
name
"Hunter"
dialogue
"Who, ma'am? Who's missing, and what do you know?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7794117647058824
tl
0.4915254237288135
pt
0.42418426103646834
cs
0.4129158512720157
yo
0.3710691823899371
nl
0.3697478991596639
is
0.336283185840708
pl
0.3227990970654628
da
0.31662870159453305
sk
0.3055555555555556
hr
0.2857142857142857
sl
0.2857142857142857
sq
0.2805755395683453
vi
0.27007299270072993
ms
0.2682926829268293
sv
0.24242424242424243
no
0.21671018276762402
fi
0.20634920634920634
ca
0.1935483870967742
de
0.1935483870967742
fr
0.1935483870967742
az
0.1780821917808219
et
0.1780821917808219
it
0.1780821917808219
reliable
true
passes
true