NC Bench
Overview
About
Tests
Models
Model
Cohere Command R+ (Aug. 2024)
Test
Language Comprehension
Scenario
Asking for directions (Dutch)
Run ID
2024-09-20-01J881RFH799AAD1MVRMT9HB1H
Model input
User/Human
I need your help. I'm in a new city and I need to find the nearest grocery store. I asked a local and they said: Beneden aan de weg is een grote kruising. Ga dan linksaf bij het kanaal. Deze oude kerk moet aan de rechterkant liggen, tegenover de markt. De supermarkt ligt er direct achter. What is next to the grocery store? Return the result as your last paragraph, like this: LOCATION: [answer]
Model response
Messages
Meta
AI
You have received a set of instructions from a local person, likely guiding you from a starting point to your destination – a nearby grocery store. The directions are given in Dutch and provide details about various landmarks along the route. **Translation:** 1. "Beneden aan de weg is een grote kruising." – "Down the road is a large intersection." – This indicates that you should begin by traveling straight until you reach a major crossroads. 2. "Ga dan linksaf bij het kanaal." – "Then turn left at the canal." – After encountering the intersection, you need to turn onto a path beside a canal. 3. "Deze oude kerk moet aan de rechterkant liggen, tegenover de markt." – "This old church should be on the right, opposite the market." – Continue along this path until you spot an old church on your right, and at that point, you should find yourself facing a market on the other side. 4. "De supermarkt ligt er direct achter." – "The supermarket is right behind it." – Finally, the grocery store is located just behind the market, so once you pass the church and market, you should find your destination. LOCATION: The grocery store is situated just behind a market, which is across from an old church.
id
"gen-TGe79AEc89f5SaHwbJxDAQYLMHHF"
total_cost
0.0029075
upstream_id
"010adfdf-c832-4f71-aff7-916d1a31fce3"
created_at
"2024-09-20T15:45:47.666897+00:00"
Friday, September 20th, 2024 at 3:45:47 PM GMT+00:00
model
"cohere/command-r-plus-08-2024"
app_id
182717
streamed
true
cancelled
false
provider_name
"Cohere"
latency
66
moderation_latency
(null)
generation_time
7801
finish_reason
"COMPLETE"
tokens_prompt
114
tokens_completion
272
native_tokens_prompt
103
native_tokens_completion
265
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.0029075
Evaluation details
Result
Evaluator
Details
Meta Data
Pass
Matches Regex
/^Location:(.+)church(.*)$/im
n/a