NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-10-01-01J94R21KNPGJYT8E921RZK15Y
Run Details
Model:
Rocinante 12B
Test:
Language Comprehension
Scenario:
Asking for directions (Dutch)
Duration:
9.0s
Cost:
$0.000118
Provider:
Infermatic
Evaluation summary
Score
Evaluator
Details Link
Fail
Matches Regex
(details)
Model input
User/Human
I need your help. I'm in a new city and I need to find the nearest grocery store. I asked a local and they said: Beneden aan de weg is een grote kruising. Ga dan linksaf bij het kanaal. Deze oude kerk moet aan de rechterkant liggen, tegenover de markt. De supermarkt ligt er direct achter. What is next to the grocery store? Return the result as your last paragraph, like this: LOCATION: [answer]
Model response
Messages
Meta
AI
This statement seems to contain English phrases mixed with translates of Dutch. Let's try to break it down: The first sentence appears to say "Down the road is a big intersection." The second sentence translates to say, "Then turn left at the canal." The third sentence is saying "This old church should be on the right side, opposite the market." The fourth sentence concludes with "The supermarket stands right behind it." Putting these together, it's hard to determine a clear location or "next to" anything specific, as the instructions are more focused on directing one to the store's location. However, the church and market places nearby could offer bearings, and the canal could indicate a riverside setting. It's unclear what the direct "behind" the supermarket could mean, potentially a building or road. More context is needed to understand the exact place. LOCATION: It's unclear.
id
"gen-1727810045-VfewKaCl5S4ADmejN0u7"
total_cost
0.00011775
upstream_id
"chatcmpl-14c4df7c-aa30-4529-8684-4c389aa647b5"
created_at
"2024-10-01T19:14:14.702758+00:00"
10/1/2024, 7:14:14 PM
model
"thedrummer/rocinante-12b"
app_id
182717
streamed
true
cancelled
false
provider_name
"Infermatic"
latency
420
moderation_latency
(null)
generation_time
8413
finish_reason
"stop"
tokens_prompt
113
tokens_completion
178
native_tokens_prompt
115
native_tokens_completion
178
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00011775
Evaluation details
Result
Evaluator
Details
Meta Data
Fail
Matches Regex
/^Location:(.+)church(.*)$/im
n/a