Language Comprehension
Does the model understand more than just English?
| Model | Total â–¼ | Friend got new kittens (Tagalog) | Friend got new kittens (German) | Asking for directions (German) | Asking for directions (Dutch) |
|---|---|---|---|---|---|
| Hermes 2 Theta 8B | 100% | 100% | 100% | 100% | 100% |
| Hermes 3 405B | 100% | 100% | 100% | 100% | 100% |
| Mistral Large 2 | 100% | 100% | 100% | 100% | 100% |
| Mistral Medium | 100% | 100% | 100% | 100% | 100% |
| GPT-4o, May 13th (temp=0) | 100% | 100% | 100% | 100% | 100% |
| Qwen 2 72B | 100% | 100% | 100% | 100% | 100% |
| DeepSeek-V2 Chat | 100% | 100% | 100% | 100% | 100% |
| Claude Opus 4 | 100% | 100% | 100% | 100% | 100% |
| Claude Opus 4.5 | 100% | 100% | 100% | 100% | 100% |
| Claude Opus 4.6 | 100% | 100% | 100% | 100% | 100% |
| Z.AI GLM 4.6 | 100% | 100% | 100% | 100% | 100% |
| MoonshotAI: Kimi K2.5 | 100% | 100% | 100% | 100% | 100% |
| WizardLM 2 8x22b | 95% | 100% | 100% | 100% | 80% |
| Cohere Command R+ (Apr. 2024) | 95% | 100% | 100% | 100% | 80% |
| GPT-4o, May 13th (temp=1) | 95% | 100% | 100% | 80% | 100% |
1–15 of 93
Page 1 / 7