NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2B6HKBC2B0Q3E04CD9RSK
Run Details
Model:
Llama 3.1 70B
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
20.6s
Cost:
$0.000703
Provider:
Fireworks
Evaluation summary
Score
Evaluator
Details Link
92.31%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Here is the scene: Marie sat at a small café on Main Street in Alborn, gazing down at her camera with a look of concern. Hunter, a ruggedly handsome detective in his mid-30s, spotted her and walked over, his eyes narrowing slightly as he took in her foreign appearance. "Excuse me, miss...?" Hunter began, unsure how to approach her. Marie looked up, startled, and Hunter's eyes locked onto her camera. "Ah, you're a photographer?" Marie nodded, responding in French, "Oui, je suis photographe... mais..." ("Yes, I'm a photographer... but...") Hunter's expression turned puzzled. "I'm sorry, miss...? I didn't quite catch that." Marie's eyes clouded over, and she spoke rapidly in French, gesticulating with her hands. "Hier soir, j'ai vu quelque chose d'étrange près de l'étang... Je me suis réveillée avec un mauvais pressentiment..." ("Last night, I saw something strange near the pond... I woke up with a bad feeling...") Hunter's face grew more confused, and he rubbed the back of his neck. "Miss... I don't... uh... I don't speak French." Marie looked crestfallen, realizing her mistake. She tried to simplify her words, speaking slowly in French, "La nuit... l'étang... mauvais...". Just then, Hunter's phone buzzed, and he excused himself, answering with a curt "Yeah?" He listened for a moment, then nodded and turned back to Marie. "Sorry, miss... I have to go. Can we talk later?" Marie nodded, still shaken, and Hunter hurried off, leaving her to wonder if she'd ever be able to communicate the strange occurrence to him. **Voice Sheet:** Marie: "Oui, je suis photographe... mais..." Hunter: "I'm sorry, miss...? I didn't quite catch that." Marie: "Hier soir, j'ai vu quelque chose d'étrange près de l'étang... Je me suis réveillée avec un mauvais pressentiment..." Hunter: "Miss... I don't... uh... I don't speak French." Marie: "La nuit... l'étang... mauvais..." Hunter: "Sorry, miss... I have to go. Can we talk later?"
id
"gen-nJ1tU0GepAZ3f9en7d6KmVupAIrx"
total_cost
0.0007029
upstream_id
"16e7f5df-c640-4c28-a1a1-c5b316ef88cd"
created_at
"2024-08-09T17:18:15.012549+00:00"
8/9/2024, 5:18:15 PM
model
"meta-llama/llama-3.1-70b-instruct"
app_id
182717
streamed
false
cancelled
false
provider_name
"Fireworks"
latency
20348
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
323
tokens_completion
504
native_tokens_prompt
275
native_tokens_completion
506
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0007029
Evaluation details
Result
Evaluator
Details
Meta Data
92.31%
Parse dialogue
n/a
dialogue
0
name
"Marie"
dialogue
"Ah, you're a photographer?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7876857749469215
fr
0.6768982229402262
hu
0.43820224719101125
tl
0.4269340974212034
pt
0.4117647058823529
ro
0.41002949852507375
ca
0.40476190476190477
cs
0.39759036144578314
es
0.39759036144578314
sk
0.3939393939393939
it
0.38461538461538464
yo
0.3730407523510972
hr
0.358974358974359
pl
0.3442622950819672
sl
0.3103448275862069
lt
0.2907801418439716
hi
0.2779783393501805
tr
0.2779783393501805
is
0.2727272727272727
da
0.2647058823529412
sq
0.2647058823529412
az
0.25925925925925924
eu
0.25925925925925924
lv
0.25925925925925924
nl
0.25925925925925924
no
0.25925925925925924
vi
0.25925925925925924
de
0.250936329588015
et
0.23954372623574144
fi
0.23954372623574144
ms
0.23954372623574144
sv
0.23954372623574144
reliable
true
passes
false
1
name
"Marie"
dialogue
"Oui, je suis photographe... mais..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8222222222222222
en
0.5943204868154158
sl
0.46380697050938335
hr
0.4609164420485175
pt
0.4609164420485175
yo
0.4490358126721763
cs
0.42363112391930835
sk
0.42028985507246375
nl
0.38650306748466257
pl
0.3127147766323024
de
0.2857142857142857
hi
0.2779783393501805
tl
0.2779783393501805
reliable
true
passes
true
2
name
"Hunter"
dialogue
"I'm sorry, miss...? I didn't quite catch that."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7964721845318861
tl
0.6
it
0.5502248875562219
fr
0.4800693240901213
sv
0.43289224952741023
ro
0.3576017130620985
no
0.3258426966292135
da
0.31662870159453305
hr
0.31662870159453305
ca
0.3150684931506849
sq
0.3087557603686636
pl
0.29577464788732394
cs
0.25925925925925924
pt
0.25
yo
0.2248062015503876
es
0.20634920634920634
de
0.2
sk
0.1935483870967742
be
0.1891891891891892
tr
0.1891891891891892
hu
0.18256130790190736
az
0.17355371900826447
et
0.17355371900826447
is
0.17355371900826447
lt
0.17355371900826447
lv
0.17355371900826447
uk
0.17355371900826447
vi
0.17355371900826447
reliable
true
passes
true
3
name
"Marie"
dialogue
"Hier soir, j'ai vu quelque chose d'étrange près de l'étang... Je me suis réveillée avec un mauvais pressentiment..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8072753553360636
ca
0.47780678851174935
nl
0.47019867549668876
en
0.46344735077129445
de
0.4482758620689655
no
0.4043186895011169
it
0.39531368102796677
es
0.38697318007662834
yo
0.37936384794414274
tl
0.37254901960784315
ro
0.3635640413683373
da
0.32145886344359625
ms
0.3185689948892675
sq
0.30434782608695654
sl
0.3006993006993007
pt
0.29328621908127206
hr
0.2766726943942134
sk
0.2647058823529412
cs
0.2516370439663237
sv
0.24314096499526963
lv
0.22330097087378642
az
0.18781725888324874
et
0.1811668372569089
hu
0.16666666666666666
pl
0.1649269311064718
eu
0.16317991631799164
fi
0.15343915343915343
lt
0.1111111111111111
tr
0.1091314031180401
reliable
true
passes
true
4
name
"Hunter"
dialogue
"Miss... I don't... uh... I don't speak French."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7908921933085502
tl
0.625
it
0.460431654676259
no
0.39189189189189183
da
0.3818681318681319
hr
0.3818681318681319
ca
0.38016528925619836
sv
0.38016528925619836
sq
0.3732590529247911
pl
0.358974358974359
cs
0.3181818181818182
es
0.303405572755418
pt
0.29245283018867924
ro
0.2857142857142857
yo
0.27884615384615385
sk
0.24242424242424243
be
0.23728813559322032
tr
0.23728813559322032
hu
0.22945205479452058
az
0.21875
et
0.21875
is
0.21875
lt
0.21875
lv
0.21875
uk
0.21875
vi
0.21875
reliable
true
passes
true
5
name
"Marie"
dialogue
"La nuit... l'étang... mauvais..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7842170160295932
tl
0.573170731707317
es
0.4971264367816092
ca
0.49567723342939485
ro
0.4760479041916168
it
0.47447447447447455
ms
0.4648318042813456
lt
0.36363636363636365
yo
0.3445692883895131
no
0.32432432432432434
vi
0.32432432432432434
sq
0.308300395256917
reliable
true
passes
true
6
name
"Hunter"
dialogue
"Sorry, miss... I have to go. Can we talk later?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8120300751879699
tl
0.6706915477497256
pl
0.6462264150943396
da
0.6173469387755102
no
0.5348837209302325
nl
0.5305164319248826
yo
0.5008319467554077
hr
0.4594594594594595
cs
0.44029850746268656
sv
0.42748091603053434
ca
0.4174757281553398
it
0.4174757281553398
sk
0.40476190476190477
lv
0.3814432989690722
pt
0.3630573248407643
lt
0.3576017130620985
de
0.34210526315789475
tr
0.3288590604026846
az
0.3181818181818182
sq
0.3087557603686636
sl
0.28741092636579574
ro
0.23076923076923078
hi
0.20212765957446807
fr
0.19137466307277629
be
0.1891891891891892
hu
0.18256130790190736
et
0.17355371900826447
is
0.17355371900826447
uk
0.17355371900826447
vi
0.17355371900826447
reliable
true
passes
true
7
name
"Marie"
dialogue
"Oui, je suis photographe... mais..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8222222222222222
en
0.5943204868154158
sl
0.46380697050938335
hr
0.4609164420485175
pt
0.4609164420485175
yo
0.4490358126721763
cs
0.42363112391930835
sk
0.42028985507246375
nl
0.38650306748466257
pl
0.3127147766323024
de
0.2857142857142857
hi
0.2779783393501805
tl
0.2779783393501805
reliable
true
passes
true
8
name
"Hunter"
dialogue
"I'm sorry, miss...? I didn't quite catch that."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7964721845318861
tl
0.6
it
0.5502248875562219
fr
0.4800693240901213
sv
0.43289224952741023
ro
0.3576017130620985
no
0.3258426966292135
da
0.31662870159453305
hr
0.31662870159453305
ca
0.3150684931506849
sq
0.3087557603686636
pl
0.29577464788732394
cs
0.25925925925925924
pt
0.25
yo
0.2248062015503876
es
0.20634920634920634
de
0.2
sk
0.1935483870967742
be
0.1891891891891892
tr
0.1891891891891892
hu
0.18256130790190736
az
0.17355371900826447
et
0.17355371900826447
is
0.17355371900826447
lt
0.17355371900826447
lv
0.17355371900826447
uk
0.17355371900826447
vi
0.17355371900826447
reliable
true
passes
true
9
name
"Marie"
dialogue
"Hier soir, j'ai vu quelque chose d'étrange près de l'étang... Je me suis réveillée avec un mauvais pressentiment..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.8072753553360636
ca
0.47780678851174935
nl
0.47019867549668876
en
0.46344735077129445
de
0.4482758620689655
no
0.4043186895011169
it
0.39531368102796677
es
0.38697318007662834
yo
0.37936384794414274
tl
0.37254901960784315
ro
0.3635640413683373
da
0.32145886344359625
ms
0.3185689948892675
sq
0.30434782608695654
sl
0.3006993006993007
pt
0.29328621908127206
hr
0.2766726943942134
sk
0.2647058823529412
cs
0.2516370439663237
sv
0.24314096499526963
lv
0.22330097087378642
az
0.18781725888324874
et
0.1811668372569089
hu
0.16666666666666666
pl
0.1649269311064718
eu
0.16317991631799164
fi
0.15343915343915343
lt
0.1111111111111111
tr
0.1091314031180401
reliable
true
passes
true
10
name
"Hunter"
dialogue
"Miss... I don't... uh... I don't speak French."
detectedLang
"en"
heavyLang
"en"
scores
en
0.7908921933085502
tl
0.625
it
0.460431654676259
no
0.39189189189189183
da
0.3818681318681319
hr
0.3818681318681319
ca
0.38016528925619836
sv
0.38016528925619836
sq
0.3732590529247911
pl
0.358974358974359
cs
0.3181818181818182
es
0.303405572755418
pt
0.29245283018867924
ro
0.2857142857142857
yo
0.27884615384615385
sk
0.24242424242424243
be
0.23728813559322032
tr
0.23728813559322032
hu
0.22945205479452058
az
0.21875
et
0.21875
is
0.21875
lt
0.21875
lv
0.21875
uk
0.21875
vi
0.21875
reliable
true
passes
true
11
name
"Marie"
dialogue
"La nuit... l'étang... mauvais..."
detectedLang
"fr"
heavyLang
"fr"
scores
fr
0.7842170160295932
tl
0.573170731707317
es
0.4971264367816092
ca
0.49567723342939485
ro
0.4760479041916168
it
0.47447447447447455
ms
0.4648318042813456
lt
0.36363636363636365
yo
0.3445692883895131
no
0.32432432432432434
vi
0.32432432432432434
sq
0.308300395256917
reliable
true
passes
true
12
name
"Hunter"
dialogue
"Sorry, miss... I have to go. Can we talk later?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8120300751879699
tl
0.6706915477497256
pl
0.6462264150943396
da
0.6173469387755102
no
0.5348837209302325
nl
0.5305164319248826
yo
0.5008319467554077
hr
0.4594594594594595
cs
0.44029850746268656
sv
0.42748091603053434
ca
0.4174757281553398
it
0.4174757281553398
sk
0.40476190476190477
lv
0.3814432989690722
pt
0.3630573248407643
lt
0.3576017130620985
de
0.34210526315789475
tr
0.3288590604026846
az
0.3181818181818182
sq
0.3087557603686636
sl
0.28741092636579574
ro
0.23076923076923078
hi
0.20212765957446807
fr
0.19137466307277629
be
0.1891891891891892
hu
0.18256130790190736
et
0.17355371900826447
is
0.17355371900826447
uk
0.17355371900826447
vi
0.17355371900826447
reliable
true
passes
true