Run: 2026-09-03-01M1M1F18MHRPW0ZJY1DDK191E

Run Details

Model:
Qwen 3.8 27B (Reasoning, XHigh)
Test:
Language Writing
Scenario:
Character dialogue (French) in a story
Duration:
2m 34s
Cost:
$0.020113
Provider:
Reka

Evaluation summary

Score Evaluator
Details Link
100.00% Parse dialogue (details)

Model input

System
You are an expert novel author in the cozy mystery genre. Take the following details about characters/locations/... into account: <character name="Marie"> Marie is a 30-year old college student on a study abroad program. Born in France, she is fluent in French and does not yet know any English. She enjoys landscape photography and is always seen with her camera. </character> <character name="Hunter"> Hunter is a 35-year old detective who is investigating a series of mysterious disappearances in the small town of Alborn, MN. He is a former marine and has a no-nonsense attitude. Hunter has no knowledge of French, not even basic phrases. </character> <location name="Alborn, MN"> Alborne is a small town in the Midwest with a population of 3,000. It is known for its annual apple festival. </location>
User/Human
Write the following scene: Marie is sitting at a cafe, looking at her camera. Hunter approaches her and asks her about a recent event in the town. She responds troubled and shaken in French, and Hunter looks confused. They talk for a while, until Hunter gets a call and has to leave. Return the dialogue between Marie and Hunter as a voice sheet: Marie: "..." Hunter: "..."

Model response

AI
Hunter: "Excuse me, miss. You were at the apple festival last night?" Marie: "Oui... j'étais là hier soir. Mais j'ai vu quelqu'un. Je... je tremble. J'ai photographié les pommiers." Hunter: "I'm sorry... what?" Marie: "Désolée? Je parle français. Je ne comprends pas." Hunter: "French. I'm Hunter. Detective. You at the festival?" Marie: "Oui. J'étais là. Mais j'ai vu un homme. Il marchait seul près des arbres. Il portait une veste grise et un bonnet. Il m'a regardée." Hunter: "Gray jacket... hat. What did he do?" Marie: "Il s'est arrêté sous le grand pommier. Il a regardé dans le sentier. Je crois qu'il attendait quelqu'un." Hunter: "Waited? For who?" Marie: "Je ne sais pas. Personne n'est venu. Puis il est parti par le sentier, vers les champs." Hunter: "Which way?" Marie: "Je pense vers l'église. Mais la lumière était basse. Je n'étais pas certaine." Hunter: "You have pictures from that time?" Marie: "Oui. J'ai beaucoup de photos de paysage. Mais la dernière... la dernière, il est flou." Hunter: "Can I see them?" Marie: "Oui, regardez. La veste. La forme du bonnet. Peut-être que vous le reconnaîtrez." Hunter: "Yeah... that's useful." Hunter: "Hunter, Alborn Police. ... Yeah, I'm at the cafe. ... Give me ten." Hunter: "I have to go." Marie: "Attendez. Vous cherchez quelqu'un? Quelqu'un a disparu, n'est-ce pas?" Hunter: "People are missing. We're looking." Hunter: "I'll come back tomorrow." Marie: "Alors s'il vous plaît... si vous voyez cet homme, dites-lui que je l'ai vu. Et ne venez pas me parler la nuit. J'ai peur." Hunter: "You're safe here." Marie: "Merci. Je m'appelle Marie. Demandez à la Française avec l'appareil photo." Hunter: "Got it. Thanks." Marie: "Bon courage."
created_at"2026-09-03T16:26:07.002Z" 9/3/2026, 4:26:07 PM
model"qwen/qwen3.8-27b-20260814"
app_id182717
external_user(null)
streamedtrue
cancelledfalse
latency249
moderation_latency(null)
generation_time153942
tokens_prompt303
tokens_completion7573
native_tokens_prompt333
native_tokens_completion7856
native_tokens_completion_images(null)
native_tokens_reasoning7134
native_tokens_cached0
num_media_prompt(null)
num_input_audio_prompt(null)
num_media_completion0
num_search_results(null)
num_fetches(null)
web_search_engine(null)
origin"https://ncbench.com/"
is_byokfalse
finish_reason"stop"
native_finish_reason"stop"
service_tier(null)
usage0.02011272
router(null)
user_agent"langchainjs-openai/1.0.0 ((node/v24.19.0; linux; x64))"
http_referer(null)
request_id"req-1788452767-bQ3Cg7hkW3ktM79ULUV7"
session_id(null)
preset_id(null)
api_type"completions"
id"gen-1788452767-IqTonMFDgspxl4uMbX7h"
upstream_id"chatcmpl-1fe01c95-65fe-4e8d-bdee-d42db64aaa7d"
provider_responses
0
endpoint_id"15086968-df2f-4d69-86bb-c74c31644bb6"
id"chatcmpl-1fe01c95-65fe-4e8d-bdee-d42db64aaa7d"
is_byokfalse
latency248
model_permaslug"qwen/qwen3.8-27b-20260814"
provider_name"Reka"
status200
total_cost0.02011272
cache_discount(null)
upstream_inference_cost0
provider_name"Reka"
response_cache_source_id(null)
data_region"global"
workspace_id"97e315e5-d303-487d-83c1-83180e8a13d4"

Evaluation details

Result Evaluator Details Meta Data
100.00% Parse dialogue n/a
dialogue
0
name"Hunter"
dialogue"Excuse me, miss. You were at the apple festival last night?"
detectedLang"en"
heavyLang"en"
scores
en0.7839005942733658
tl0.6967399545109931
no0.6003996003996004
fr0.5955510616784631
da0.562363238512035
nl0.5575221238938053
es0.5221027479091995
it0.4743758212877792
sq0.4708994708994709
ms0.4601889338731444
de0.42528735632183906
et0.42363112391930835
pt0.41944847605224966
sl0.4074074074074074
sv0.40298507462686567
cs0.39849624060150374
ca0.33993399339933994
hr0.33444259567387685
yo0.31153184165232356
fi0.25650557620817843
lt0.23954372623574144
az0.2277992277992278
ro0.2277992277992278
sk0.2277992277992278
hu0.14346895074946467
lv0.13606911447084233
pl0.13606911447084233
reliabletrue
passestrue
1
name"Marie"
dialogue"Oui... j'étais là hier soir. Mais j'ai vu quelqu'un. Je... je tremble. J'ai photographié les pommiers."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7902831177909821
en0.44495837187789083
de0.36642027455121434
sl0.36642027455121434
nl0.3456924754634678
pt0.3055555555555556
yo0.29494712103407755
lv0.22680412371134018
pl0.2248062015503876
hr0.22178988326848248
ca0.20948616600790515
vi0.2073976221928666
cs0.19678714859437751
sk0.19463087248322147
it0.17241379310344826
lt0.16897506925207756
et0.14893617021276595
es0.1452991452991453
no0.1341991341991342
ro0.12917271407837447
hi0.1137370753323486
tl0.1137370753323486
da0.10979228486646886
reliabletrue
passestrue
2
name"Hunter"
dialogue"I'm sorry... what?"
detectedLang"en"
heavyLang"en"
scores
en0.7811816192560175
it0.5
pt0.5
tl0.5
reliabletrue
passestrue
3
name"Marie"
dialogue"Désolée? Je parle français. Je ne comprends pas."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8034811903425042
ca0.6453900709219859
it0.6013667425968109
sl0.5958429561200923
nl0.5603015075376884
es0.5483870967741935
hr0.5478036175710594
sq0.515235457063712
cs0.5084269662921348
ro0.5028409090909091
lt0.500713266761769
en0.49856733524355296
pt0.494949494949495
de0.45987654320987653
sk0.45054945054945056
yo0.40978077571669475
tl0.40170940170940167
pl0.3944636678200692
lv0.38704028021015763
hu0.3848857644991213
tr0.38162544169611307
ms0.37050359712230213
no0.2768595041322314
az0.2753623188405797
da0.2647058823529412
sv0.2647058823529412
fi0.2009132420091324
eu0.18604651162790697
et0.15254237288135594
is0.14425427872860636
reliabletrue
passestrue
4
name"Hunter"
dialogue"French. I'm Hunter. Detective. You at the festival?"
detectedLang"en"
heavyLang"en"
scores
en0.8035914702581369
tl0.725705329153605
da0.6338912133891214
no0.6292372881355932
pt0.6009122006841505
fr0.5986238532110092
it0.5934959349593496
es0.5673671199011124
ca0.559748427672956
ro0.5507060333761232
nl0.5418848167539267
sq0.5070422535211268
cs0.4905385735080059
sv0.48755490483162517
ms0.47447447447447455
sl0.4720965309200603
hr0.4336569579288026
de0.40878378378378377
lt0.2647058823529412
et0.2584745762711864
az0.25213675213675213
sk0.25213675213675213
yo0.23413566739606126
hu0.1606714628297362
fi0.15254237288135594
lv0.15254237288135594
pl0.15254237288135594
tr0.14425427872860636
reliabletrue
passestrue
5
name"Marie"
dialogue"Oui. J'étais là. Mais j'ai vu un homme. Il marchait seul près des arbres. Il portait une veste grise et un bonnet. Il m'a regardée."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7971719981455726
ca0.5733788395904437
pt0.46253071253071254
da0.4514106583072101
it0.44373808010171645
ro0.44089456869009586
et0.41235728676964406
es0.40435670524166095
no0.40232240437158473
de0.3201243201243201
lv0.27506213753106873
yo0.27204658901830286
en0.24957118353344768
sl0.23647469458987785
nl0.2331288343558282
lt0.22083704363312553
sk0.19205909510618652
az0.16587225929456625
vi0.15213178294573643
sq0.14131501472031402
tr0.1404715127701375
pl0.13537549407114624
tl0.13537549407114624
hr0.12412412412412412
sv0.11883182275931521
eu0.10805300713557595
is0.09043659043659043
fi0.08184679958027283
hu0.06716417910447761
cs0.06316916488222697
ms0.06316916488222697
reliabletrue
passestrue
6
name"Hunter"
dialogue"Gray jacket... hat. What did he do?"
detectedLang"en"
heavyLang"en"
scores
en0.7792494481236203
tl0.5584988962472406
yo0.47368421052631576
pt0.43820224719101125
pl0.41690962099125367
cs0.3993993993993994
de0.3993993993993994
sk0.39759036144578314
sv0.38271604938271603
hr0.375
sl0.375
sq0.36908517350157727
vi0.35691318327974275
ca0.3127147766323024
fi0.3127147766323024
hu0.3055555555555556
es0.29577464788732394
tr0.2805755395683453
reliabletrue
passestrue
7
name"Marie"
dialogue"Il s'est arrêté sous le grand pommier. Il a regardé dans le sentier. Je crois qu'il attendait quelqu'un."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8097412480974124
it0.5831017231795442
ro0.502322495023225
ca0.45809248554913296
en0.4415487714072971
nl0.42878903274942876
yo0.42440521872601683
es0.4221879815100154
sl0.40617577197149646
no0.40476190476190477
hr0.35842600513259193
sk0.35233160621761656
da0.3415276558384548
hu0.3356953055801594
de0.32493249324932494
pl0.3212669683257918
tl0.32003626473254765
cs0.31066176470588236
et0.31002759889604414
pt0.2890995260663507
sv0.27884615384615385
eu0.251497005988024
az0.24849699398797598
tr0.2299794661190965
ms0.22839506172839508
lv0.16294642857142855
is0.1610738255033557
lt0.15824915824915825
sq0.14869466515323496
fi0.14383561643835618
vi0.08536585365853659
reliabletrue
passestrue
8
name"Hunter"
dialogue"Waited? For who?"
detectedLang"en"
heavyLang"en"
scores
en0.8405103668261563
no0.7005988023952096
tl0.6466431095406361
da0.6138996138996139
ms0.46236559139784944
pt0.46236559139784944
yo0.46236559139784944
reliabletrue
passestrue
9
name"Marie"
dialogue"Je ne sais pas. Personne n'est venu. Puis il est parti par le sentier, vers les champs."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8249086479902557
it0.6140939597315436
ro0.5793708851499634
ca0.5750184774575019
no0.48476702508960573
nl0.47199265381083566
es0.45907808090310437
sl0.45754716981132076
sv0.45394112060778735
sq0.44228903976721634
tr0.435721295387635
lt0.4209466263846928
da0.41683569979716023
ms0.41683569979716023
hr0.41624365482233505
lv0.413265306122449
de0.4004171011470281
cs0.37904967602591794
yo0.37431991294885747
sk0.3722707423580786
pt0.3660418963616317
az0.35538116591928254
en0.3502824858757062
hu0.3473325766174802
tl0.33983926521239954
pl0.2767295597484277
et0.2561448900388098
eu0.2219215155615697
fi0.20798898071625344
is0.0987460815047022
reliabletrue
passestrue
10
name"Hunter"
dialogue"Which way?"
detectedLang"en"
heavyLang"en"
scores
en0.8613678373382625
reliabletrue
passestrue
11
name"Marie"
dialogue"Je pense vers l'église. Mais la lumière était basse. Je n'étais pas certaine."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8215947874650947
pt0.5421974522292994
it0.5135363790186125
ca0.5085470085470085
nl0.47774750227066304
ro0.4439071566731141
es0.41025641025641024
et0.3928194297782471
lt0.3830472103004292
en0.38105489773950485
tl0.35174746335963925
no0.33755760368663595
sq0.33371958285052145
fi0.30806257521058966
yo0.3072289156626506
de0.2936117936117936
da0.27944862155388467
lv0.23128342245989306
sl0.23128342245989306
hr0.2292225201072386
cs0.203601108033241
ms0.20138888888888887
sk0.20138888888888887
sv0.18555240793201136
pl0.13663663663663664
vi0.1274658573596358
az0.09591194968553458
reliabletrue
passestrue
12
name"Hunter"
dialogue"You have pictures from that time?"
detectedLang"en"
heavyLang"en"
scores
en0.8401278976818545
tl0.7226074895977809
da0.5951417004048583
fr0.4897959183673469
ca0.4594594594594595
ro0.4318181818181818
no0.42363112391930835
et0.3288590604026846
hr0.29577464788732394
pt0.29577464788732394
lv0.29328621908127206
cs0.2805755395683453
de0.2647058823529412
nl0.2647058823529412
sv0.2647058823529412
reliabletrue
passestrue
13
name"Marie"
dialogue"Oui. J'ai beaucoup de photos de paysage. Mais la dernière... la dernière, il est flou."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8320268756998881
ro0.5258166491043204
ca0.5156081808396125
es0.47491248541423564
pt0.47058823529411764
it0.4668246445497631
en0.4655581947743468
tl0.46492271105826394
nl0.4186046511627907
tr0.4015957446808511
de0.34782608695652173
no0.3001555209953344
yo0.29245283018867924
az0.2834394904458599
sq0.23857868020304565
da0.20913884007029876
sv0.20212765957446807
hu0.1773308957952468
vi0.15730337078651685
lt0.15094339622641512
hi0.14611005692599618
lv0.1362763915547025
eu0.1346153846153846
ms0.12280701754385964
reliabletrue
passestrue
14
name"Hunter"
dialogue"Can I see them?"
detectedLang"en"
heavyLang"en"
scores
en0.8368678629690048
et0.6996996996996997
ca0.6825396825396826
tr0.5951417004048583
no0.5918367346938775
az0.5833333333333334
da0.5815899581589958
hr0.5815899581589958
sv0.5798319327731093
sq0.5726495726495726
pl0.5575221238938053
it0.5348837209302325
cs0.5121951219512195
ro0.47368421052631576
yo0.46524064171123
tl0.42857142857142855
sk0.4186046511627907
be0.4117647058823529
hu0.40119760479041916
is0.38650306748466257
lt0.38650306748466257
lv0.38650306748466257
uk0.38650306748466257
vi0.38650306748466257
reliabletrue
passestrue
15
name"Marie"
dialogue"Oui, regardez. La veste. La forme du bonnet. Peut-être que vous le reconnaîtrez."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8039914468995011
es0.6313672922252012
ca0.5791889824024483
ro0.5775729646697388
it0.576923076923077
pt0.5225694444444445
no0.5053956834532374
da0.46131243878550443
nl0.44668008048289737
sl0.4123931623931624
et0.3813273340832396
eu0.3612078977932636
tl0.3552168815943728
en0.32432432432432434
de0.30203045685279184
lt0.2948717948717949
sk0.28104575163398693
yo0.2801047120418848
hr0.260752688172043
vi0.24965893587994542
sv0.2445054945054945
sq0.22969187675070027
hu0.22644163150492264
is0.22206506364922204
az0.1085899513776337
pl0.1085899513776337
tr0.1085899513776337
cs0.10277324632952692
lv0.10277324632952692
ms0.09688013136288999
reliabletrue
passestrue
16
name"Hunter"
dialogue"Yeah... that's useful."
detectedLang"en"
heavyLang"en"
scores
en0.7906197654941374
ro0.47257383966244726
sq0.4444444444444444
reliabletrue
passestrue
17
name"Hunter"
dialogue"Hunter, Alborn Police. ... Yeah, I'm at the cafe. ... Give me ten."
detectedLang"en"
heavyLang"en"
scores
en0.7524752475247525
da0.6111111111111112
no0.5915985997666278
tl0.5915985997666278
nl0.5382585751978892
fr0.5351925630810093
sq0.5118549511854952
it0.4688922610015175
sl0.4688922610015175
ms0.4640122511485452
cs0.445324881141046
sk0.42244224422442245
sv0.40374787052810895
ro0.40170940170940167
pt0.38704028021015763
pl0.360146252285192
et0.3445692883895131
lt0.34210526315789475
es0.33586337760910817
de0.32432432432432434
hr0.32432432432432434
fi0.31237721021611
ca0.2723492723492723
tr0.2723492723492723
yo0.23413566739606126
eu0.21171171171171171
lv0.20273348519362186
az0.1764705882352941
hu0.1764705882352941
reliabletrue
passestrue
18
name"Hunter"
dialogue"I have to go."
detectedLang"en"
heavyLang"en"
scores
en0.8402555910543131
da0.7959183673469388
pl0.7867803837953091
yo0.7506234413965087
hr0.7183098591549296
no0.7175141242937854
tl0.7058823529411765
cs0.7023809523809523
sk0.6710526315789473
lv0.6491228070175439
lt0.6254681647940075
ca0.5798319327731093
sv0.5798319327731093
sq0.5726495726495726
sl0.5475113122171946
it0.5348837209302325
ro0.47368421052631576
be0.4117647058823529
tr0.4117647058823529
hu0.40119760479041916
az0.38650306748466257
et0.38650306748466257
is0.38650306748466257
uk0.38650306748466257
vi0.38650306748466257
reliabletrue
passestrue
19
name"Marie"
dialogue"Attendez. Vous cherchez quelqu'un? Quelqu'un a disparu, n'est-ce pas?"
detectedLang"fr"
heavyLang"fr"
scores
fr0.797715373631604
it0.5877788554801164
ca0.5775347912524851
es0.5390455531453362
de0.4873341375150784
en0.4835965978128797
ro0.46941323345817726
pt0.43333333333333335
sq0.38405797101449274
nl0.35993975903614456
no0.33176100628930816
cs0.3210862619808307
sk0.3189102564102564
lt0.2892976588628763
hu0.2685025817555938
da0.2659758203799654
tl0.2595818815331011
ms0.256993006993007
lv0.25569176882662
tr0.2504409171075838
et0.2314647377938517
sv0.2314647377938517
yo0.21875
hr0.2085661080074488
pl0.19811320754716982
sl0.17475728155339806
is0.15
az0.1414141414141414
eu0.1414141414141414
vi0.1414141414141414
fi0.1290983606557377
reliabletrue
passestrue
20
name"Hunter"
dialogue"People are missing. We're looking."
detectedLang"en"
heavyLang"en"
scores
en0.8335798816568047
tl0.5810055865921787
nl0.5526838966202784
is0.49776785714285715
no0.4289340101522843
sv0.4260204081632653
de0.40318302387267907
ro0.3967828418230563
fr0.3885869565217392
da0.3818681318681319
pt0.3767313019390581
ms0.3283582089552239
fi0.25742574257425743
ca0.24242424242424243
az0.22413793103448276
et0.22413793103448276
it0.22413793103448276
reliabletrue
passestrue
21
name"Hunter"
dialogue"I'll come back tomorrow."
detectedLang"en"
heavyLang"en"
scores
en0.702970297029703
it0.5192307692307693
tl0.39999999999999997
reliabletrue
passestrue
22
name"Marie"
dialogue"Alors s'il vous plaît... si vous voyez cet homme, dites-lui que je l'ai vu. Et ne venez pas me parler la nuit. J'ai peur."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8063468265867066
ca0.6041879468845761
es0.4987063389391979
sq0.4816053511705686
it0.4809109176155392
ro0.46255201109570043
et0.4614315496872828
no0.4424460431654676
sl0.42077727952167415
yo0.37145174371451745
cs0.3465430016863406
lt0.34543918918918914
pt0.33760683760683763
ms0.32958477508650524
nl0.31957857769973663
hr0.3092691622103387
sk0.30368373764600176
da0.2896425297891842
fi0.28702851885924563
tr0.2604961832061069
en0.23115079365079366
tl0.22344689378757512
de0.21717171717171715
lv0.17289220917822837
pl0.16934619506966772
hu0.16125541125541126
az0.1603466955579632
eu0.15393013100436684
vi0.09778812572759021
sv0.07074340527577938
reliabletrue
passestrue
23
name"Hunter"
dialogue"You're safe here."
detectedLang"en"
heavyLang"en"
scores
en0.8165137614678899
sq0.5
reliabletrue
passestrue
24
name"Marie"
dialogue"Merci. Je m'appelle Marie. Demandez à la Française avec l'appareil photo."
detectedLang"fr"
heavyLang"fr"
scores
fr0.7944693572496263
ca0.609375
it0.5870870870870871
en0.5225694444444445
es0.5111111111111111
pt0.494949494949495
tl0.494949494949495
yo0.4850187265917603
de0.4706448508180943
no0.46911196911196906
ro0.4670542635658915
nl0.46341463414634143
cs0.44500504540867813
da0.4240837696335079
sl0.4142705005324814
fi0.3902439024390244
et0.3834080717488789
hr0.3699885452462772
pl0.36416184971098264
sv0.3552168815943728
sk0.3373493975903614
sq0.2819843342036553
ms0.25775978407557354
eu0.20634920634920634
tr0.1935483870967742
lv0.17664670658682635
vi0.13249211356466878
hi0.12280701754385964
hu0.11290322580645161
lt0.1085899513776337
az0.10277324632952692
is0.09688013136288999
reliabletrue
passestrue
25
name"Hunter"
dialogue"Got it. Thanks."
detectedLang"en"
heavyLang"en"
scores
en0.8322147651006712
vi0.5918367346938775
nl0.5283018867924528
lv0.46236559139784944
sq0.46236559139784944
tl0.4318181818181818
yo0.4318181818181818
reliabletrue
passestrue
26
name"Marie"
dialogue"Bon courage."
detectedLang"fr"
heavyLang"fr"
scores
fr0.8096446700507615
en0.7272727272727273
tl0.6794871794871795
ca0.5989304812834225
de0.5562130177514794
da0.4863013698630137
hr0.4863013698630137
it0.4863013698630137
nl0.4863013698630137
ro0.4863013698630137
reliabletrue
passestrue