NC Bench
Overview
About
Tests
Models
Model
MN GRAND Gutenberg Lyra4 12B Madness
Test
Language Comprehension
Scenario
Asking for directions (German)
Run ID
2024-10-16-01JAAT7AR741ZVR2Z5DXRR7CDF
Model input
User/Human
I need your help. I'm in a new city and I need to find the nearest grocery store. I asked a local and they said: Die Straße runter, da ist eine große Kreuzung. Dann an der Littfasssäule links abbiegen. Rechts sollte diese alte Kirche sein, quer über den Markt. Der Supermarkt ist direkt dahinter. What is next to the grocery store? Return the result as your last paragraph, like this: LOCATION: [answer]
Model response
Messages
Meta
AI
The key parts of the directions are: - The street down from here (Straße runter) - At the large intersection (große Kreuzung) - Turn left at the Littfasssäule (Littfasssäule links abbiegen) - The old church should be on the right (rechts sollte diese alte Kirche sein) - Across the market square (quer über den Markt) - The supermarket is right behind it (direkt dahinter) Putting it all together, you go down the street you're currently on. At the end of the block you'll hit a busy intersection with a crosswalk. Look for a large light pole or sign post there called a Littfasssäule and turn left onto the next street. You should see an old historic church building on the right side as you walk. Continue past it into a plaza or public square. The grocery store entrance will be immediately past the square, right after the church, so you can see the front of the store from the church steps. The location described as being right after the grocery store is: LOCATION: The old church<|end_header_id|>
Evaluation details
Result
Evaluator
Details
Meta Data
Pass
Matches Regex
/^Location:(.+)church(.*)$/im
n/a