Asking for directions (German)

Language Comprehension

Does the model understand more than just English?

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
GPT-5.6 Sol (Reasoning)	100%
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.6 Flash (Reasoning)	100%
Qwen3.7 Max	100%
Grok 4.5 (Reasoning, High)	100%
Gemini 3.1 Pro (Preview)	100%
GPT-5.4 (Reasoning)	100%
Muse Spark 1.1 (Reasoning, Medium)	100%
Z.AI GLM 5.1	100%
Qwen3.6 Max Preview	100%
GPT-5.5 (Reasoning)	100%
Claude Sonnet 4.6 (Reasoning)	100%
Z.AI GLM 5.2 (Reasoning, High)	100%
Gemini 3.5 Flash (Reasoning)	100%
Z.AI GLM 5 Turbo	100%
MoonshotAI: Kimi K3 (Reasoning, High)	100%
MoonshotAI: Kimi K2.6	100%
Claude Opus 4.7 (Reasoning)	100%
GPT-5.5 (Reasoning, Low)	100%
GPT-5.6 Terra (Reasoning)	100%

	Score	Cost	Time
Mistral NeMO	100%	$0.0000	321ms
Ministral 8B	60%	$0.0000	341ms
Ministral 3 3B	100%	$0.0000	691ms
GPT-4o Mini (temp=0)	100%	$0.0000	728ms
GPT-4o Mini (temp=1)	100%	$0.0000	800ms
Mistral Large 3	100%	$0.0001	7.5s
Mistral Small 3.2 24B	100%	$0.0000	1.6s
Gemini 2.5 Flash Lite	100%	$0.0001	722ms
Gemma 4 26B	100%	$0.0000	11.1s
Gemma 3 4B	100%	$0.0000	2.4s
GPT-5.4 Mini	80%	$0.0002	812ms
Cydonia 24B V4.1	100%	$0.0001	2.1s
DeepSeek V4 Flash	100%	$0.0000	20.9s
Hermes 3 70B	80%	$0.0001	3.2s
DeepSeek V3 (2025-03-24)	100%	$0.0001	7.5s
Gemini 3.1 Flash Lite	80%	$0.0002	922ms
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0002	3.5s
DeepSeek V3 (2024-12-26)	100%	$0.0001	3.2s
Z.AI GLM 4.5	80%	$0.0001	3.8s
Grok 4.3	60%	$0.0002	740ms

	Score	Cost	Speed	Stability
Mistral NeMO	100%	$0.0000	321ms	100%
Ministral 3 3B	100%	$0.0000	691ms	100%
GPT-4o Mini (temp=0)	100%	$0.0000	728ms	100%
GPT-4o Mini (temp=1)	100%	$0.0000	800ms	100%
Gemini 2.5 Flash Lite	100%	$0.0001	722ms	100%
Mistral Small 3.2 24B	100%	$0.0000	1.6s	100%
Gemma 3 4B	100%	$0.0000	2.4s	100%
Gemini 3.1 Flash Lite (Preview)	100%	$0.0002	995ms	100%
Cydonia 24B V4.1	100%	$0.0001	2.1s	100%
Gemini 3.5 Flash Lite (Reasoning, Minimal)	100%	$0.0003	693ms	100%
GPT-5.6 Luna	100%	$0.0003	709ms	100%
Gemini 3 Flash (Preview)	100%	$0.0003	1.3s	100%
Gemini 3.5 Flash Lite (Reasoning)	100%	$0.0004	740ms	100%
DeepSeek V3.1	100%	$0.0002	2.4s	100%
DeepSeek V3 (2024-12-26)	100%	$0.0001	3.2s	100%
Llama 3.1 70B	100%	$0.0002	3.3s	100%
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0002	3.5s	100%
DeepSeek-V2 Chat	100%	$0.0000	4.6s	100%
WizardLM 2 8x22b	100%	$0.0002	3.8s	100%
GPT-5.6 Terra	100%	$0.0007	1.0s	100%

Language Comprehension

Asking for directions (German)

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	Total
77	GPT-5.6 Sol (Reasoning)	$0.0054	3.7s	100%	100	100	100	100	100	100%
108	Claude Opus 4.6 (Reasoning)	$0.011	9.6s	100%	100	100	100	100	100	100%
86	Gemini 3.6 Flash (Reasoning)	$0.0069	4.4s	100%	100	100	100	100	100	100%
98	Qwen3.7 Max	$0.0072	25.1s	100%	100	100	100	100	100	100%
89	Grok 4.5 (Reasoning, High)	$0.0066	13.2s	100%	100	100	100	100	100	100%
102	Gemini 3.1 Pro (Preview)	$0.0096	12.5s	100%	100	100	100	100	100	100%
80	GPT-5.4 (Reasoning)	$0.0050	8.1s	100%	100	100	100	100	100	100%
110	Muse Spark 1.1 (Reasoning, Medium)	$0.013	20.1s	100%	100	100	100	100	100	100%
96	Z.AI GLM 5.1	$0.0039	47.8s	100%	100	100	100	100	100	100%
124	Qwen3.6 Max Preview	$0.013	50.8s	100%	100	100	100	100	100	100%
92	GPT-5.5 (Reasoning)	$0.0081	5.7s	100%	100	100	100	100	100	100%
88	Claude Sonnet 4.6 (Reasoning)	$0.0068	7.4s	100%	100	100	100	100	100	100%
67	Z.AI GLM 5.2 (Reasoning, High)	$0.0029	13.6s	100%	100	100	100	100	100	100%
84	Gemini 3.5 Flash (Reasoning)	$0.0068	3.7s	100%	100	100	100	100	100	100%
53	Z.AI GLM 5 Turbo	$0.0022	8.0s	100%	100	100	100	100	100	100%
101	MoonshotAI: Kimi K3 (Reasoning, High)	$0.0088	18.2s	100%	100	100	100	100	100	100%
146	MoonshotAI: Kimi K2.6	$0.013	2.1m	100%	100	100	100	100	100	100%
64	Claude Opus 4.7 (Reasoning)	$0.0042	2.5s	100%	100	100	100	100	100	100%
69	GPT-5.5 (Reasoning, Low)	$0.0043	5.8s	100%	100	100	100	100	100	100%
42	GPT-5.6 Terra (Reasoning)	$0.0022	2.7s	100%	100	100	100	100	100	100%
109	Claude Opus 4.8 (Reasoning)	$0.012	6.5s	100%	100	100	100	100	100	100%
70	Claude Opus 4.6	$0.0046	5.0s	100%	100	100	100	100	100	100%
78	MoonshotAI: Kimi K3 (Reasoning, Low)	$0.0046	9.8s	100%	100	100	100	100	100	100%
111	Claude Opus 5 (Reasoning, Low)	$0.015	8.5s	100%	100	100	100	100	100	100%
107	Claude Opus 4.8 (Reasoning, Low)	$0.012	5.9s	100%	100	100	100	100	100	100%
100	Muse Spark 1.1 (Reasoning, Minimal)	$0.0089	15.6s	100%	100	100	100	100	100	100%
54	GPT-5 Mini	$0.0016	12.6s	100%	100	100	100	100	100	100%
113	Claude Opus 5 (Reasoning)	$0.017	9.3s	100%	100	100	100	100	100	100%
94	Qwen 3.5 397B A17B	$0.0046	34.8s	100%	100	100	100	100	100	100%
97	Grok 4.3 (Reasoning)	$0.0062	31.7s	100%	100	100	100	100	100	100%
61	Grok 4.5 (Reasoning, Low)	$0.0033	6.2s	100%	100	100	100	100	100	100%
60	GPT-5.4 (Reasoning, Low)	$0.0033	4.2s	100%	100	100	100	100	100	100%
91	Grok 4.20 (Reasoning)	$0.0056	23.1s	100%	100	100	100	100	100	100%
106	MoonshotAI: Kimi K2.5	$0.0057	46.4s	100%	100	100	100	100	100	100%
58	Thinking Machines Inkling (Reasoning)	$0.0028	7.4s	100%	100	100	100	100	100	100%
39	GPT-5.6 Sol	$0.0020	1.4s	100%	100	100	100	100	100	100%
49	Claude Sonnet 4.6	$0.0026	3.5s	100%	100	100	100	100	100	100%
33	GPT-5.6 Luna (Reasoning)	$0.0012	3.2s	100%	100	100	100	100	100	100%
43	MiniMax M3	$0.0007	14.3s	100%	100	100	100	100	100	100%
72	Claude Sonnet 5 (Reasoning)	$0.0047	6.2s	100%	100	100	100	100	100	100%
81	Qwen 3.5 122B	$0.0046	11.7s	100%	100	100	100	100	100	100%
59	Claude Sonnet 5 (Reasoning, Low)	$0.0031	5.3s	100%	100	100	100	100	100	100%
79	Qwen 3.5 27B	$0.0038	16.2s	100%	100	100	100	100	100	100%
90	Claude Opus 5	$0.0078	6.2s	100%	100	100	100	100	100	100%
35	Gemini 3 Flash (Preview, Reasoning)	$0.0013	3.2s	100%	100	100	100	100	100	100%
75	Claude Opus 4.7	$0.0054	3.7s	100%	100	100	100	100	100	100%
45	GPT-5.4 Mini (Reasoning)	$0.0019	6.1s	100%	100	100	100	100	100	100%
76	Qwen 3.5 Plus (2026-04-20)	$0.0031	20.8s	100%	100	100	100	100	100	100%
51	Gemma 4 31B (Reasoning)	$0.0002	21.6s	100%	100	100	100	100	100	100%
63	Claude Opus 4.5	$0.0040	3.6s	100%	100	100	100	100	100	100%
87	Z.AI GLM 5	$0.0035	30.7s	100%	100	100	100	100	100	100%
55	ByteDance Seed 1.6	$0.0012	15.3s	100%	100	100	100	100	100	100%
41	GPT-5.5	$0.0021	1.7s	100%	100	100	100	100	100	100%
47	Qwen 3.6 Flash	$0.0020	6.6s	100%	100	100	100	100	100	100%
82	DeepSeek V4 Pro (Reasoning)	$0.0017	33.6s	100%	100	100	100	100	100	100%
30	Gemma 4 26B (Reasoning)	$0.0002	8.7s	100%	100	100	100	100	100	100%
85	Aion 3.0	$0.0050	17.5s	100%	100	100	100	100	100	100%
93	Gemini 2.5 Pro	$0.0078	8.0s	100%	100	100	100	100	100	100%
103	Qwen 3.6 27B	$0.0059	43.4s	100%	100	100	100	100	100	100%
20	GPT-5.6 Terra	$0.0007	1.0s	100%	100	100	100	100	100	100%
40	DeepSeek V4 Flash (Reasoning)	$0.0002	14.3s	100%	100	100	100	100	100	100%
104	Z.AI GLM 4.7	$0.0021	1.2m	100%	100	100	100	100	100	100%
56	Qwen 3.6 35B	$0.0018	11.1s	100%	100	100	100	100	100	100%
74	Z.AI GLM 4.6	$0.0020	27.6s	100%	100	100	100	100	100	100%
46	Claude Sonnet 4	$0.0023	3.1s	100%	100	100	100	100	100	100%
50	Claude Sonnet 4.5	$0.0026	3.8s	100%	100	100	100	100	100	100%
105	Claude Opus 4	$0.010	14.1s	100%	100	100	100	100	100	100%
95	Qwen 3.5 35B	$0.0064	23.4s	100%	100	100	100	100	100	100%
68	MiniMax M2.5	$0.0014	25.5s	100%	100	100	100	100	100	100%
31	Gemini 3.6 Flash (Reasoning, Minimal)	$0.0014	1.3s	100%	100	100	100	100	100	100%
52	Aion 2.0	$0.0011	16.3s	100%	100	100	100	100	100	100%
65	o4 Mini	$0.0033	9.6s	100%	100	100	100	100	100	100%
71	MiniMax M2.7	$0.0019	25.7s	100%	100	100	100	100	100	100%
23	Qwen 3.5 Plus (2026-02-15)	$0.0003	5.3s	100%	100	100	100	100	100	100%
17	Gemini 3.1 Flash Lite (Reasoning)	$0.0002	3.5s	100%	100	100	100	100	100	100%
36	Gemini 3.5 Flash (Reasoning, Minimal)	$0.0016	1.4s	100%	100	100	100	100	100	100%
112	ByteDance Seed 2.0 Mini	$0.0017	1.8m	100%	100	100	100	100	100	100%
62	Qwen 3.5 Flash	$0.0012	21.9s	100%	100	100	100	100	100	100%
12	Gemini 3 Flash (Preview)	$0.0003	1.3s	100%	100	100	100	100	100	100%
8	Gemini 3.1 Flash Lite (Preview)	$0.0002	995ms	100%	100	100	100	100	100	100%
11	GPT-5.6 Luna	$0.0003	709ms	100%	100	100	100	100	100	100%
32	Gemma 4 26B	$0.0000	11.1s	100%	100	100	100	100	100	100%
10	Gemini 3.5 Flash Lite (Reasoning, Minimal)	$0.0003	693ms	100%	100	100	100	100	100	100%
13	Gemini 3.5 Flash Lite (Reasoning)	$0.0004	740ms	100%	100	100	100	100	100	100%
37	GPT-5.4	$0.0016	2.0s	100%	100	100	100	100	100	100%
24	Mistral Large 3	$0.0001	7.5s	100%	100	100	100	100	100	100%
73	ByteDance Seed 2.0 Lite	$0.0022	25.5s	100%	100	100	100	100	100	100%
18	DeepSeek-V2 Chat	$0.0000	4.6s	100%	100	100	100	100	100	100%
83	Qwen 3.5 9B	$0.0005	46.8s	100%	100	100	100	100	100	100%
26	GPT-5.4 Mini (Reasoning, Low)	$0.0008	3.5s	100%	100	100	100	100	100	100%
27	Claude Haiku 4.5	$0.0010	2.3s	100%	100	100	100	100	100	100%
15	DeepSeek V3 (2024-12-26)	$0.0001	3.2s	100%	100	100	100	100	100	100%
14	DeepSeek V3.1	$0.0002	2.4s	100%	100	100	100	100	100	100%
22	DeepSeek V3.2	$0.0001	5.6s	100%	100	100	100	100	100	100%
66	Z.AI GLM 4.7 Flash	$0.0006	30.0s	100%	100	100	100	100	100	100%
29	DeepSeek V4 Pro	$0.0004	6.3s	100%	100	100	100	100	100	100%
48	DeepSeek V4 Flash	$0.0000	20.9s	100%	100	100	100	100	100	100%
38	Mistral Large 2	$0.0012	4.8s	100%	100	100	100	100	100	100%
34	Hermes 3 405B	$0.0000	11.8s	100%	100	100	100	100	100	100%
44	Z.AI GLM 4.5 Air	$0.0006	15.3s	100%	100	100	100	100	100	100%
28	Laguna S 2.1	$0.0001	8.4s	100%	100	100	100	100	100	100%
25	DeepSeek V3 (2025-03-24)	$0.0001	7.5s	100%	100	100	100	100	100	100%
5	Gemini 2.5 Flash Lite	$0.0001	722ms	100%	100	100	100	100	100	100%
4	GPT-4o Mini (temp=1)	$0.0000	800ms	100%	100	100	100	100	100	100%
16	Llama 3.1 70B	$0.0002	3.3s	100%	100	100	100	100	100	100%
6	Mistral Small 3.2 24B	$0.0000	1.6s	100%	100	100	100	100	100	100%
3	GPT-4o Mini (temp=0)	$0.0000	728ms	100%	100	100	100	100	100	100%
21	Gemma 3 12B	$0.0000	5.6s	100%	100	100	100	100	100	100%
99	Nemotron 3 Nano	$0.0011	1.2m	100%	100	100	100	100	100	100%
9	Cydonia 24B V4.1	$0.0001	2.1s	100%	100	100	100	100	100	100%
19	WizardLM 2 8x22b	$0.0002	3.8s	100%	100	100	100	100	100	100%
57	Cohere Command R+ (Aug. 2024)	$0.0029	5.5s	100%	100	100	100	100	100	100%
7	Gemma 3 4B	$0.0000	2.4s	100%	100	100	100	100	100	100%
2	Ministral 3 3B	$0.0000	691ms	100%	100	100	100	100	100	100%
1	Mistral NeMO	$0.0000	321ms	100%	100	100	100	100	100	100%
161	GPT-5	$0.018	29.8s	20%	100	100	100	100	0	80%
130	GPT-5.1	$0.0049	10.2s	20%	100	100	100	100	0	80%
127	GPT-5.2	$0.0025	5.2s	20%	100	100	100	100	0	80%
128	Claude Sonnet 5	$0.0031	4.7s	20%	100	100	100	100	0	80%
122	GPT-4.1	$0.0013	2.4s	20%	100	100	100	100	0	80%
125	Xiaomi MIMO v2.5 Pro	$0.0012	8.3s	20%	100	100	100	100	0	80%
118	Gemma 4 31B	$0.0000	4.0s	20%	100	100	100	100	0	80%
115	Gemini 3.1 Flash Lite	$0.0002	922ms	20%	100	100	100	100	0	80%
119	Z.AI GLM 4.5	$0.0001	3.8s	20%	100	100	100	100	0	80%
126	GPT-OSS 120B	$0.0003	16.1s	20%	100	100	100	100	0	80%
131	Gemini 2.5 Flash (Reasoning)	$0.0051	10.0s	20%	100	100	100	100	0	80%
129	Aion 3.0 Mini	$0.0014	27.6s	20%	100	100	100	100	0	80%
123	Gemini 2.5 Flash Lite (Reasoning)	$0.0007	6.8s	20%	100	100	100	100	0	80%
117	Gemini 2.5 Flash	$0.0004	1.2s	20%	100	100	100	100	0	80%
114	GPT-5.4 Mini	$0.0002	812ms	20%	100	100	100	100	0	80%
121	Qwen 3 32B	$0.0001	5.8s	20%	100	100	100	100	0	80%
120	Qwen 2.5 72B	$0.0001	4.3s	20%	100	100	100	100	0	80%
116	Hermes 3 70B	$0.0001	3.2s	20%	100	100	100	100	0	80%
148	o4 Mini High	$0.0051	13.6s	2%	100	100	100	0	0	60%
137	Xiaomi MIMO v2.5	$0.0007	4.1s	2%	100	100	100	0	0	60%
134	Inception Mercury 2	$0.0003	709ms	2%	100	100	100	0	0	60%
135	Mistral Small 4 (Reasoning)	$0.0004	4.6s	2%	100	100	100	0	0	60%
133	Grok 4.3	$0.0002	740ms	2%	100	100	100	0	0	60%
138	Gemma 3 27B	$0.0001	9.1s	2%	100	100	100	0	0	60%
136	Laguna XS 2.1	$0.0001	7.3s	2%	100	100	100	0	0	60%
132	Ministral 8B	$0.0000	341ms	2%	100	100	100	0	0	60%
142	GPT-4.1 Mini	$0.0003	2.4s	0%	100	100	0	0	0	40%
144	GPT-4o, Aug. 6th (temp=1)	$0.0008	1.3s	0%	100	100	0	0	0	40%
140	Grok 4.20	$0.0003	871ms	0%	100	100	0	0	0	40%
147	GPT-5 Nano	$0.0006	17.6s	0%	100	100	0	0	0	40%
145	Qwen3 235B A22B Instruct 2507	$0.0002	12.6s	0%	100	100	0	0	0	40%
141	GPT-5.4 Nano (Reasoning, Low)	$0.0002	2.5s	0%	100	100	0	0	0	40%
139	GPT-5.4 Nano	$0.0001	1.1s	0%	100	100	0	0	0	40%
143	ByteDance Seed 1.6 Flash	$0.0002	4.1s	0%	100	100	0	0	0	40%
160	Arcee AI: Trinity Mini	$0.0001	1.0m	0%	100	100	0	0	0	40%
153	Nemotron 3 Super	$0.0000	13.7s	0%	100	0	0	0	0	20%
152	GPT-5.4 Nano (Reasoning)	$0.0003	4.5s	0%	100	0	0	0	0	20%
154	Writer: Palmyra X5	$0.0019	9.9s	0%	100	0	0	0	0	20%
150	Mistral Small 4	$0.0001	1.2s	0%	100	0	0	0	0	20%
151	GPT-4.1 Nano	$0.0001	1.5s	0%	100	0	0	0	0	20%
149	Ministral 3B	$0.0000	405ms	0%	100	0	0	0	0	20%
159	GPT-4o, Aug. 6th (temp=0)	$0.0009	2.2s	0%	0	0	0	0	0	0%
158	Mistral Medium 3.1	$0.0005	4.5s	0%	0	0	0	0	0	0%
157	Thinking Machines Inkling	$0.0004	4.4s	0%	0	0	0	0	0	0%
156	Ministral 3 14B	$0.0000	1.0s	0%	0	0	0	0	0	0%
155	Ministral 3 8B	$0.0000	693ms	0%	0	0	0	0	0	0%
86.34%