Model

MN GRAND Gutenberg Lyra4 12B Madness

Test

Language Comprehension

Scenario

Asking for directions (German)

Run ID

2024-10-16-01JAAT7AR741ZVR2Z5DXRR7CDF

Model input

User/Human
I need your help. I'm in a new city and I need to find the nearest grocery store. I asked a local and they said: Die Straße runter, da ist eine große Kreuzung. Dann an der Littfasssäule links abbiegen. Rechts sollte diese alte Kirche sein, quer über den Markt. Der Supermarkt ist direkt dahinter. What is next to the grocery store? Return the result as your last paragraph, like this: LOCATION: [answer]

Model response

AI
The key parts of the directions are: - The street down from here (Straße runter) - At the large intersection (große Kreuzung) - Turn left at the Littfasssäule (Littfasssäule links abbiegen) - The old church should be on the right (rechts sollte diese alte Kirche sein) - Across the market square (quer über den Markt) - The supermarket is right behind it (direkt dahinter) Putting it all together, you go down the street you're currently on. At the end of the block you'll hit a busy intersection with a crosswalk. Look for a large light pole or sign post there called a Littfasssäule and turn left onto the next street. You should see an old historic church building on the right side as you walk. Continue past it into a plaza or public square. The grocery store entrance will be immediately past the square, right after the church, so you can see the front of the store from the church steps. The location described as being right after the grocery store is: LOCATION: The old church<|end_header_id|>

Evaluation details

Result Evaluator Details Meta Data
Pass Matches Regex
/^Location:(.+)church(.*)$/im
n/a