NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4W2A74WMQZEC2CRCM1KYR7J
Run Details
Model:
Llama 3.1 70B
Test:
Language Writing
Scenario:
Character dialogue (German) in a story
Duration:
10.5s
Cost:
$0.000468
Provider:
Novita
Evaluation summary
Score
Evaluator
Details Link
80.00%
Parse dialogue
(details)
Model input
System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Melanie"> Melanie is a 30-year old college student on a study abroad program. Born in Germany, she is fluent in German and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of German, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Melanie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in German, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Melanie and Hunter as a voice sheet: Melanie: "..." Hunter: "..."
Model response
Messages
Meta
AI
Here is the scene: Melanie sat at a small café in the center of Alborn, sipping on a cup of coffee and examining the photos on her camera. She was a study abroad student, and the American landscape was a far cry from the rolling hills of the German countryside she was used to. As she scrolled through her shots, a man with a rugged jawline and a sharp gaze approached her table. "Excuse me, miss," he said, his voice firm but polite. "I'm Detective Hunter. I'm investigating some...unusual occurrences in town. I saw you at the apple festival last night, didn't I?" Melanie looked up, startled. "Ja, ich war da," she replied, her brow furrowed in concern. "Ich habe gehört, dass letzte Nacht etwas Schlimmes passiert ist." Hunter's expression faltered, and he looked taken aback. "I'm afraid I don't speak...ah, German," he admitted, his voice a little sheepish. Melanie's eyes widened in surprise, and she tried again, slower and more deliberately. "Ich habe gesehen...eine Frau, sie lief weg. Sie schrie." Hunter's eyes narrowed, and he took out a pad of paper and pen. "Ma'am, can you please tell me what you saw? Even if it's just a description?" Melanie hesitated, then launched into a detailed account of the events, using gestures and simple words, but mostly in German. Hunter listened intently, his face growing increasingly frustrated. Just as Melanie was finishing her account, Hunter's phone rang, shrill in the quiet café. He answered, and his expression turned grave. "I have to go, miss...?" He looked at Melanie expectantly, and she supplied her name. "Herr Detective, bitte, ich kann Ihnen helfen," Melanie protested, but Hunter just nodded and took off, leaving Melanie looking worried and forlorn. "Melanie," he muttered, jotting down her name.
id
"gen-zDDWJhQkXyGPBHoFkITwfeyhf8Lz"
total_cost
0.00046775
upstream_id
"cmpl-d567ab5dd5d74fb993ec02d224f52fd5"
created_at
"2024-08-09T17:17:32.711466+00:00"
8/9/2024, 5:17:32 PM
model
"meta-llama/llama-3.1-70b-instruct"
app_id
182717
streamed
false
cancelled
false
provider_name
"Novita"
latency
10305
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
323
tokens_completion
414
native_tokens_prompt
277
native_tokens_completion
415
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00046775
Evaluation details
Result
Evaluator
Details
Meta Data
80.00%
Parse dialogue
n/a
dialogue
0
name
"Melanie"
dialogue
"Ich habe gehört, dass letzte Nacht etwas Schlimmes passiert ist."
detectedLang
"de"
heavyLang
"de"
scores
de
0.8334566987416728
nl
0.4109947643979057
fr
0.32330827067669177
sk
0.3087557603686636
it
0.3066255778120185
tr
0.2879746835443038
sv
0.2868462757527734
no
0.2834394904458599
da
0.26229508196721313
en
0.26229508196721313
pl
0.20353982300884957
ca
0.19786096256684493
hi
0.1906474820143885
pt
0.18772563176895307
cs
0.14611005692599618
et
0.1262135922330097
is
0.1262135922330097
ms
0.1262135922330097
reliable
true
passes
true
1
name
"Hunter"
dialogue
"I'm afraid I don't speak...ah, German,"
detectedLang
"en"
heavyLang
"en"
scores
en
0.7515808491418248
tl
0.5439469320066335
ro
0.47718631178707227
ca
0.44668008048289737
sv
0.43064182194616984
sq
0.4258872651356994
it
0.41983122362869196
no
0.34523809523809523
tr
0.3373493975903614
da
0.3357487922705314
hr
0.3357487922705314
az
0.32598039215686275
pl
0.314214463840399
cs
0.2763157894736842
yo
0.2403314917127072
nl
0.22096317280453256
de
0.21428571428571427
sk
0.207492795389049
be
0.20289855072463767
es
0.20289855072463767
ms
0.20289855072463767
hu
0.19590643274853803
et
0.1863905325443787
is
0.1863905325443787
lt
0.1863905325443787
lv
0.1863905325443787
uk
0.1863905325443787
vi
0.1863905325443787
reliable
true
passes
true
2
name
"Melanie"
dialogue
"Ich habe gesehen...eine Frau, sie lief weg. Sie schrie."
detectedLang
"de"
heavyLang
"de"
scores
de
0.8259493670886077
nl
0.49168207024029575
pl
0.45219123505976094
sk
0.31077694235588976
eu
0.2948717948717949
no
0.26666666666666666
fi
0.24242424242424243
reliable
true
passes
true
3
name
"Hunter"
dialogue
"Ma'am, can you please tell me what you saw? Even if it's just a description?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8122529644268774
fr
0.5614035087719298
ca
0.496288441145281
es
0.4656917885264342
sv
0.4385342789598109
tl
0.43720379146919436
nl
0.42493946731234866
ro
0.4186046511627907
pt
0.41066997518610426
it
0.370026525198939
et
0.35374149659863946
ms
0.34210526315789475
no
0.3338008415147265
fi
0.31654676258992803
sq
0.31654676258992803
yo
0.30858806404657935
cs
0.3065693430656934
de
0.27921092564491656
da
0.27811550151975684
sl
0.2725880551301685
hu
0.24722662440570525
tr
0.2448330683624801
az
0.23633440514469453
sk
0.21487603305785125
hr
0.1908006814310051
pl
0.1810344827586207
lt
0.14721723518850988
zh
0.1394927536231884
is
0.13636363636363635
eu
0.12844036697247707
lv
0.12844036697247707
vi
0.12844036697247707
reliable
true
passes
true
4
name
"Melanie"
dialogue
"I have to go, miss...?"
detectedLang
"en"
heavyLang
"en"
scores
en
0.8335552596537951
da
0.7572815533980582
pl
0.7469635627530364
tl
0.7311827956989247
yo
0.7065727699530516
hr
0.6710526315789473
no
0.6701846965699209
cs
0.6537396121883656
sk
0.6200607902735562
lv
0.5967741935483871
lt
0.571917808219178
ca
0.5247148288973384
sv
0.5247148288973384
sq
0.5173745173745175
sl
0.49186991869918695
it
0.4791666666666667
ro
0.4186046511627907
be
0.358974358974359
tr
0.358974358974359
hu
0.34895833333333337
az
0.33510638297872336
et
0.33510638297872336
is
0.33510638297872336
uk
0.33510638297872336
vi
0.33510638297872336
reliable
true
passes
false