Run: 2026-09-03-01M1KZ1XSGKBMHXKPH8JPHD01W

Run Details

Model:
Qwen 3.8 27B (Reasoning, XHigh)
Test:
Language Writing
Scenario:
Character dialogue (German) in a story
Duration:
2m 38s
Cost:
$0.025985
Provider:
Phala

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Melanie"> Melanie is a 30-year old college student on a study abroad program. Born in Germany, she is fluent in German and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of German, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Melanie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in German, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Melanie and Hunter as a voice sheet: Melanie: "..." Hunter: "..."

Model response

AI
Melanie: "Ja?" Hunter: "Ma'am, Detective Hunter. I'm investigating the disappearances around the apple festival. Did you see anything that looked wrong?" Melanie: "Entschuldigung? Ich... ich verstehe nichts. Ich spreche kein Englisch." Hunter: "Right. I'm sorry. I don't speak German. I don't understand a word." Melanie: "Es tut mir leid. Ich bin so aufgewühlt. Ich habe etwas gesehen. Bitte, Sie müssen helfen." Hunter: "Okay. I'm going to use my phone translator. It will ask in German, and I'll listen to what you say." Melanie: "Ja. Bitte." Hunter: "The phone is asking now. Ma'am, whatever it asks, just answer in German." Melanie: "Ich habe die Frau am See gesehen. Sie stand allein. Ein Mann kam näher, und dann..." Hunter: "My phone says: 'You saw the woman by the lake. She was alone. A man came closer, and then...'" Melanie: "Ja. Dann war es dunkel. Ich hatte die Kamera in der Hand, aber..." Hunter: "Your camera. Did you record anything?" Melanie: "Ja. Aber meine Hände haben gezittert. Das Bild ist nicht gut. Bitte, nicht jetzt." Hunter: "I need to see it, ma'am. It could be important." Melanie: "Es ist... es ist zu viel. Ich brauche einen Moment." Hunter: "Understood. I have to take a call. I'll be back in a few minutes, and I'll bring someone who speaks German." Melanie: "Jemand, der Deutsch spricht? Ja. Bitte. War es die Frau vom Festival?" Hunter: "I'll check. Stay right here. Don't go anywhere." Melanie: "Ich bleibe hier. Ja. Sie kommen zurück?" Hunter: "I'll be right back."
created_at"2026-09-03T15:44:00.313Z" 9/3/2026, 3:44:00 PM
model"qwen/qwen3.8-27b-20260814"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency614
moderation_latency(null)
generation_time157942
tokens_prompt306
tokens_completion8721
native_tokens_prompt405
native_tokens_completion8615
native_tokens_completion_images(null)
native_tokens_reasoning8193
native_tokens_cached64
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.0259846
router(null)
user_agent"langchainjs-openai/1.0.0 ((node/v24.19.0; linux; x64))"
http_referer(null)
request_id"req-1788450240-k15yyHUhjbAVmGjzEcIz"
session_id(null)
preset_id(null)
api_type"completions"
id"gen-1788450240-QOkJPZ5YeX9wqFnriw6c"
upstream_id"req_a86e3b8dfaa09912d9674f2261ad9603"
provider_responses
0
endpoint_id"cfc4d0ca-df7c-4b5f-9c74-328c41a7cbc3"
id"req_a86e3b8dfaa09912d9674f2261ad9603"
is_byokfalse
latency614
model_permaslug"qwen/qwen3.8-27b-20260814"
provider_name"Phala"
status200
total_cost0.0259846
cache_discount0.0000224
upstream_inference_cost0
provider_name"Phala"
response_cache_source_id(null)
data_region"global"
workspace_id"97e315e5-d303-487d-83c1-83180e8a13d4"

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Melanie"
dialogue"Ja?"
detectedLang"et"
heavyLang""
scores
et0.8603351955307262
fi0.8595505617977528
lv0.8214285714285714
ca0.8134328358208955
de0.7787610619469026
sk0.7706422018348624
da0.7619047619047619
pl0.7619047619047619
sl0.7619047619047619
sq0.75
sv0.75
yo0.75
reliablefalse
passestrue
1
name"Hunter"
dialogue"Ma'am, Detective Hunter. I'm investigating the disappearances around the apple festival. Did you see anything that looked wrong?"
detectedLang"en"
heavyLang"en"
scores
en0.7931509997701678
tl0.6216897856242118
fr0.5750708215297451
pt0.4983277591973244
nl0.47826086956521735
it0.4770482277745497
es0.4548758328285887
ca0.4522215459525259
ro0.45054945054945056
ms0.44886711573790566
no0.4163424124513619
sv0.40357852882703776
de0.39999999999999997
sq0.3910690121786198
da0.3822923816060398
hr0.3440233236151603
cs0.34162399414776884
sl0.3278566094100075
et0.3186979560938683
sk0.2214532871972318
lt0.21875
az0.2077464788732394
lv0.18699186991869918
vi0.14611005692599618
is0.13294797687861273
fi0.11937377690802348
yo0.10625620655412116
hu0.06928645294725957
pl0.06542056074766354
tr0.06152241918665276
reliabletrue
passestrue
2
name"Melanie"
dialogue"Entschuldigung? Ich... ich verstehe nichts. Ich spreche kein Englisch."
detectedLang"de"
heavyLang"de"
scores
de0.8130224373075232
nl0.6320346320346321
ro0.35703479576399394
no0.3431221020092735
en0.3410852713178295
sk0.3122977346278317
ms0.30213464696223313
lt0.28209459459459457
hr0.2595818815331011
da0.24911660777385158
it0.24911660777385158
is0.24242424242424243
sv0.23561151079136694
pl0.21296296296296297
sl0.20560747663551404
cs0.1904761904761905
hu0.1448692152917505
es0.1290983606557377
fr0.1290983606557377
pt0.1290983606557377
az0.121900826446281
ca0.121900826446281
sq0.121900826446281
reliabletrue
passestrue
3
name"Hunter"
dialogue"Right. I'm sorry. I don't speak German. I don't understand a word."
detectedLang"en"
heavyLang"en"
scores
en0.8022335970218706
tl0.6446488294314382
da0.579624134520277
no0.579624134520277
sv0.5314222712238148
ro0.5219347581552306
it0.49941107184923444
de0.4904076738609112
ca0.4572158365261813
nl0.43408788282290284
cs0.43029490616621985
pl0.4209809264305177
sk0.4064245810055866
yo0.39716312056737585
sq0.39372325249643364
hr0.371301775147929
pt0.3609022556390977
hu0.3441358024691358
tr0.33800623052959505
is0.3328100470957614
az0.32324840764331214
es0.32216905901116427
fr0.31006493506493504
lt0.25438596491228066
lv0.23835125448028674
ms0.23835125448028674
vi0.23835125448028674
et0.22867513611615245
sl0.17475728155339806
be0.1414141414141414
eu0.1414141414141414
fi0.1290983606557377
uk0.1290983606557377
reliabletrue
passestrue
4
name"Melanie"
dialogue"Es tut mir leid. Ich bin so aufgewühlt. Ich habe etwas gesehen. Bitte, Sie müssen helfen."
detectedLang"de"
heavyLang"de"
scores
de0.8278958395689914
nl0.3575418994413408
no0.32113341204250295
sk0.2857142857142857
pl0.2830423940149626
tr0.2404227212681638
da0.2374005305039788
sl0.20798898071625344
ca0.203601108033241
es0.203601108033241
yo0.1797432239657632
az0.17503586800573886
lv0.1726618705035971
eu0.16666666666666666
en0.16424418604651161
et0.16424418604651161
hu0.15066469719350073
hr0.14814814814814814
sv0.14814814814814814
vi0.14688427299703263
lt0.13663663663663664
fi0.13273001508295626
ms0.13273001508295626
tl0.13273001508295626
it0.11538461538461538
reliabletrue
passestrue
5
name"Hunter"
dialogue"Okay. I'm going to use my phone translator. It will ask in German, and I'll listen to what you say."
detectedLang"en"
heavyLang"en"
scores
en0.8062015503875969
tl0.6616134271792096
de0.5363501483679526
nl0.4703389830508474
ro0.45746527777777773
pl0.4517543859649123
da0.4318181818181818
sv0.42502299908003677
sl0.42075996292863765
cs0.4001919385796545
no0.39671814671814676
yo0.3902439024390244
sq0.3693239152371342
lv0.36612576064908725
it0.3609406952965235
sk0.35233160621761656
hr0.33510638297872336
ms0.29775280898876405
fr0.2664319248826291
vi0.26122931442080377
az0.2541766109785203
es0.251497005988024
pt0.2460796139927624
tr0.24242424242424243
fi0.23124231242312424
eu0.22744128553770088
lt0.21482412060301506
ca0.2028061224489796
et0.1610738255033557
hu0.11347517730496455
is0.10071942446043164
reliabletrue
passestrue
6
name"Melanie"
dialogue"Ja. Bitte."
detectedLang"de"
heavyLang"de"
scores
de0.8106060606060606
et0.7807017543859649
da0.673913043478261
fi0.6710526315789473
lv0.6052631578947368
ca0.592391304347826
nl0.5283018867924528
sk0.5283018867924528
pl0.5161290322580645
sl0.5161290322580645
it0.5
no0.5
sq0.5
sv0.5
yo0.5
reliabletrue
passestrue
7
name"Hunter"
dialogue"The phone is asking now. Ma'am, whatever it asks, just answer in German."
detectedLang"en"
heavyLang"en"
scores
en0.7947454844006568
nl0.5974235104669887
tl0.5777027027027027
sv0.5078740157480316
yo0.43883277216610556
de0.4298745724059293
fr0.37888198757763975
ro0.37733499377334995
no0.3498049414824447
da0.3288590604026846
sq0.31880108991825606
it0.3160054719562244
ms0.3084370677731674
is0.24585218702865763
ca0.24357034795764
az0.23547400611620795
hu0.23547400611620795
sl0.23547400611620795
pt0.2223950233281493
vi0.21011058451816747
et0.16943521594684383
lv0.14675767918088736
pl0.14383561643835618
es0.12280701754385964
tr0.12280701754385964
reliabletrue
passestrue
8
name"Melanie"
dialogue"Ich habe die Frau am See gesehen. Sie stand allein. Ein Mann kam näher, und dann..."
detectedLang"de"
heavyLang"de"
scores
de0.8235709244883558
nl0.51171875
no0.4897959183673469
en0.47589098532494756
et0.4165694282380397
pl0.38271604938271603
is0.38042131350681535
sk0.35064935064935066
ro0.34895833333333337
da0.34725848563968675
fi0.3429697766097241
fr0.31412894375857336
it0.24127465857359634
cs0.2401215805471125
sl0.2295839753466872
lt0.21011058451816747
sq0.21011058451816747
tl0.20508744038155804
eu0.18699186991869918
lv0.15110356536502548
yo0.14965986394557826
hi0.1482112436115843
hr0.14089347079037798
tr0.11816578483245149
az0.11190053285968028
sv0.11190053285968028
ms0.10554561717352416
reliabletrue
passestrue
9
name"Hunter"
dialogue"My phone says: 'You saw the woman by the lake. She was alone. A man came closer, and then...'"
detectedLang"en"
heavyLang"en"
scores
en0.8129472999349381
tl0.5432883240667197
nl0.4815148782687105
da0.4739249771271729
fr0.4390243902439025
cs0.4221105527638191
yo0.41742654508611954
pl0.4053774560496381
de0.3960084033613445
sk0.37971952535059333
no0.3646408839779005
sv0.3179122182680902
ms0.2953431372549019
sq0.2776381909547738
ro0.2758186397984887
it0.26937738246505716
vi0.2609254498714653
hr0.2414248021108179
lv0.23638778220451526
lt0.23435419440745675
hu0.213406292749658
fi0.1991643454038997
pt0.1958041958041958
ca0.19127988748241914
es0.18670438472418668
sl0.13533834586466165
zh0.11944869831546708
tr0.11809815950920245
is0.11538461538461538
az0.10852713178294573
eu0.10852713178294573
et0.0987460815047022
reliabletrue
passestrue
10
name"Melanie"
dialogue"Ja. Dann war es dunkel. Ich hatte die Kamera in der Hand, aber..."
detectedLang"de"
heavyLang"de"
scores
de0.8325659271661783
nl0.623352165725047
sv0.5575221238938053
da0.5459704880817253
no0.4784876140808344
en0.47506561679790027
ca0.47229551451187335
sl0.4310099573257468
it0.42363112391930835
pl0.4108983799705449
sk0.4091580502215657
yo0.38556067588325654
es0.38080495356037153
tl0.37791601866251945
lv0.3700787401574803
ms0.35064935064935066
ro0.3485342019543974
tr0.28186714542190305
et0.2779783393501805
fi0.2766726943942134
hu0.20318725099601595
lt0.18533604887983707
az0.17355371900826447
fr0.17355371900826447
cs0.16142557651991615
hr0.16142557651991615
sq0.15789473684210525
reliabletrue
passestrue
11
name"Hunter"
dialogue"Your camera. Did you record anything?"
detectedLang"en"
heavyLang"en"
scores
en0.7987117552334944
tl0.6621621621621622
ro0.5659722222222222
ca0.5503597122302158
it0.5462794918330308
es0.5274102079395085
fr0.5107632093933464
ms0.4769874476987448
nl0.4669509594882729
pt0.2857142857142857
sq0.2857142857142857
sv0.24698795180722888
cs0.2378048780487805
de0.22360248447204967
reliabletrue
passestrue
12
name"Melanie"
dialogue"Ja. Aber meine Hände haben gezittert. Das Bild ist nicht gut. Bitte, nicht jetzt."
detectedLang"de"
heavyLang"de"
scores
de0.8346456692913385
nl0.49567723342939485
no0.4637385086823289
sv0.4348762109795479
fi0.4101123595505618
et0.4061085972850679
tr0.35424354243542433
ca0.26264044943820225
es0.25213675213675213
pt0.2402315484804631
da0.22794117647058826
lv0.1796875
hi0.16798732171156894
cs0.16
fr0.14910858995137766
sk0.13793103448275862
pl0.13223140495867766
sl0.13223140495867766
is0.12935323383084577
en0.1264559068219634
it0.125
sq0.125
yo0.125
reliabletrue
passestrue
13
name"Hunter"
dialogue"I need to see it, ma'am. It could be important."
detectedLang"en"
heavyLang"en"
scores
en0.8131109833237493
tl0.5932415519399249
ca0.5729303547963206
fr0.5590230664857531
yo0.5578231292517007
no0.5467224546722455
da0.5396600566572238
et0.5337159253945482
ro0.5289855072463767
lt0.5185185185185186
it0.4881889763779528
sq0.468954248366013
lv0.4546979865771812
pl0.44539249146757676
hr0.43965517241379315
cs0.4206773618538324
sv0.38679245283018865
sk0.385633270321361
es0.37739463601532564
pt0.37739463601532564
hu0.36149312377210213
tr0.3157894736842105
sl0.2713004484304933
nl0.1792929292929293
be0.17721518987341772
az0.16237113402061853
is0.16237113402061853
uk0.16237113402061853
vi0.16237113402061853
reliabletrue
passestrue
14
name"Melanie"
dialogue"Es ist... es ist zu viel. Ich brauche einen Moment."
detectedLang"de"
heavyLang"de"
scores
de0.8405797101449275
ca0.5634920634920635
nl0.5585874799357945
fr0.544701986754967
sk0.4850187265917603
pl0.48015122873345933
es0.4607843137254902
lv0.45
en0.3834080717488789
ro0.3820224719101124
hu0.375
fi0.358974358974359
no0.3574766355140187
sq0.3357487922705314
cs0.3176178660049628
eu0.2838541666666667
et0.2819843342036553
hi0.27821522309711283
tr0.25474254742547425
lt0.2486338797814208
is0.2403314917127072
tl0.2403314917127072
it0.23611111111111113
pt0.23611111111111113
da0.19590643274853803
az0.1863905325443787
hr0.1863905325443787
sv0.1863905325443787
yo0.1863905325443787
reliabletrue
passestrue
15
name"Hunter"
dialogue"Understood. I have to take a call. I'll be back in a few minutes, and I'll bring someone who speaks German."
detectedLang"en"
heavyLang"en"
scores
en0.8192436040044493
tl0.6304718590108015
da0.5878249841471148
sv0.5657982631930528
no0.5252008765522279
de0.5178041543026707
nl0.5072024260803639
yo0.496124031007752
ro0.48083067092651754
pl0.47622884770346496
it0.45286195286195285
cs0.4477485131690739
sl0.44206008583690987
hr0.43868739205526774
ca0.43771626297577854
sk0.4337979094076655
lt0.39478584729981375
es0.3930905695611578
et0.385633270321361
ms0.3557978196233895
pt0.35323383084577115
hu0.34343434343434337
sq0.34277047522750254
az0.3062966915688367
fr0.3055555555555556
lv0.281767955801105
tr0.2502883506343714
is0.1751269035532995
fi0.17197452229299362
vi0.1698595146871009
be0.09722222222222222
eu0.09722222222222222
uk0.08835904628330996
reliabletrue
passestrue
16
name"Melanie"
dialogue"Jemand, der Deutsch spricht? Ja. Bitte. War es die Frau vom Festival?"
detectedLang"de"
heavyLang"de"
scores
de0.8172959805115714
nl0.6481626270523847
da0.5985727029438002
ca0.5553359683794467
en0.504950495049505
no0.5022123893805309
ro0.46808510638297873
et0.4636471990464839
lv0.45520581113801456
es0.4230769230769231
sv0.38101788170563966
it0.3775933609958506
sk0.3767313019390581
hr0.37586685159500693
ms0.3723849372384937
sl0.3723849372384937
fr0.36079545454545453
yo0.3580599144079886
lt0.3253373313343328
fi0.32432432432432434
sq0.31297709923664124
pt0.3087557603686636
cs0.293563579277865
tl0.28457869634340227
hu0.2730210016155089
pl0.24114671163575044
az0.2077464788732394
reliabletrue
passestrue
17
name"Hunter"
dialogue"I'll check. Stay right here. Don't go anywhere."
detectedLang"en"
heavyLang"en"
scores
en0.796875
tl0.5952677459526774
sq0.3538767395626242
de0.3186582809224318
yo0.3172268907563025
pl0.2494226327944573
da0.18546365914786966
fr0.18546365914786966
no0.18546365914786966
reliabletrue
passestrue
18
name"Melanie"
dialogue"Ich bleibe hier. Ja. Sie kommen zurück?"
detectedLang"de"
heavyLang"de"
scores
de0.8356344510190664
da0.5682210708117443
pl0.5511669658886894
sv0.5421245421245421
et0.5145631067961165
fi0.5088408644400786
no0.48559670781893005
sk0.4541484716157206
nl0.43820224719101125
lv0.3150684931506849
ca0.30362116991643456
fr0.26035502958579887
hu0.2492492492492493
sl0.24242424242424243
sq0.23076923076923078
yo0.23076923076923078
reliabletrue
passestrue
19
name"Hunter"
dialogue"I'll be right back."
detectedLang"en"
heavyLang"en"
scores
en0.8100303951367781
tl0.7065727699530516
yo0.5954692556634305
hu0.4834710743801653
lt0.47257383966244726
da0.3718592964824121
de0.3718592964824121
fr0.3718592964824121
no0.3718592964824121
reliabletrue
passestrue