Location rename: market square, outer ring, bridge, northern mines

Text Replacement

Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.

Text Editing

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
GPT-5.6 Sol (Reasoning)	100%
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.6 Flash (Reasoning)	100%
Qwen3.7 Max	100%
Grok 4.5 (Reasoning, High)	100%
GPT-5.4 (Reasoning)	100%
Muse Spark 1.1 (Reasoning, Medium)	100%
Z.AI GLM 5.1	100%
Qwen3.6 Max Preview	100%
GPT-5.5 (Reasoning)	100%
Claude Sonnet 4.6 (Reasoning)	100%
Z.AI GLM 5.2 (Reasoning, High)	100%
Gemini 3.5 Flash (Reasoning)	100%
Z.AI GLM 5 Turbo	100%
MoonshotAI: Kimi K3 (Reasoning, High)	100%
MoonshotAI: Kimi K2.6	100%
Claude Opus 4.7 (Reasoning)	100%
GPT-5.5 (Reasoning, Low)	100%
GPT-5.6 Terra (Reasoning)	100%
Claude Opus 4.8 (Reasoning)	100%

	Score	Cost	Time
Ministral 3B	100%	$0.0000	1.9s
Ministral 3 3B	100%	$0.0001	1.9s
Ministral 8B	100%	$0.0001	3.2s
Ministral 3 8B	100%	$0.0002	3.1s
Gemini 2.5 Flash Lite	100%	$0.0003	1.5s
Gemma 3 4B	100%	$0.0001	5.8s
Ministral 3 14B	100%	$0.0002	3.8s
Mistral Small 3.2 24B	100%	$0.0002	4.5s
Gemma 3 12B	100%	$0.0001	8.5s
Laguna S 2.1	100%	$0.0002	7.8s
DeepSeek V4 Flash	100%	$0.0002	8.2s
Mistral Small 4	100%	$0.0004	2.8s
Laguna XS 2.1	100%	$0.0002	8.7s
Gemma 4 26B	100%	$0.0003	15.4s
Qwen 2.5 72B	100%	$0.0003	10.4s
GPT-4o Mini (temp=1)	100%	$0.0004	9.5s
GPT-5.4 Nano (Reasoning, Low)	100%	$0.0007	2.8s
GPT-5.4 Nano	100%	$0.0007	3.0s
Gemma 3 27B	100%	$0.0002	13.2s
GPT-5.4 Nano (Reasoning)	100%	$0.0009	4.3s

	Score	Cost	Speed	Stability
Ministral 3B	100%	$0.0000	1.9s	100%
Ministral 3 3B	100%	$0.0001	1.9s	100%
Gemini 2.5 Flash Lite	100%	$0.0003	1.5s	100%
Ministral 8B	100%	$0.0001	3.2s	100%
Ministral 3 8B	100%	$0.0002	3.1s	100%
Mistral Small 4	100%	$0.0004	2.8s	100%
Ministral 3 14B	100%	$0.0002	3.8s	100%
Mistral Small 3.2 24B	100%	$0.0002	4.5s	100%
Gemini 3.1 Flash Lite (Preview)	100%	$0.0009	1.6s	100%
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0009	1.7s	100%
Gemma 3 4B	100%	$0.0001	5.8s	100%
Gemini 3.1 Flash Lite	100%	$0.0009	1.9s	100%
GPT-5.4 Nano (Reasoning, Low)	100%	$0.0007	2.8s	100%
GPT-5.4 Nano	100%	$0.0007	3.0s	100%
Gemini 3.5 Flash Lite (Reasoning, Minimal)	100%	$0.0014	1.4s	100%
Gemini 3.5 Flash Lite (Reasoning)	100%	$0.0014	1.4s	100%
Laguna S 2.1	100%	$0.0002	7.8s	100%
GPT-5.4 Nano (Reasoning)	100%	$0.0009	4.3s	100%
Gemma 3 12B	100%	$0.0001	8.5s	100%
Gemini 2.5 Flash	100%	$0.0014	2.0s	100%

Median	Evaluator	Top 3	Flop 3
100.0%	Location replacement accuracy	100Gemini 3.6 Flash (Reasoning) 100GPT-OSS 120B 100Qwen 3.5 35B	52Hermes 3 70B 88GPT-4.1 Nano 88Mistral NeMO
100.0%	No remaining old location names	100GPT-5.6 Luna (Reasoning) 100DeepSeek V4 Pro (Reasoning) 100Claude Opus 5 (Reasoning)	43Hermes 3 70B
100.0%	Non-location text preserved	100Qwen 3.6 35B 100Qwen 3.6 Flash 100Qwen 2.5 72B	57Hermes 3 70B 92Gemini 3.1 Pro (Preview) 92Nemotron 3 Nano

Text Replacement

Location rename: market square, outer ring, bridge, northern mines

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	Total
124	GPT-5.6 Sol (Reasoning)	$0.018	4.8s	100%	100	100	100	100	100	100	100	100%
148	Claude Opus 4.6 (Reasoning)	$0.026	12.4s	100%	100	100	100	100	100	100	100	100%
121	Gemini 3.6 Flash (Reasoning)	$0.017	7.6s	100%	100	100	100	100	100	100	100	100%
154	Qwen3.7 Max	$0.028	48.5s	100%	100	100	100	100	100	100	100	100%
116	Grok 4.5 (Reasoning, High)	$0.014	17.4s	100%	100	100	100	100	100	100	100	100%
108	GPT-5.4 (Reasoning)	$0.012	9.0s	100%	100	100	100	100	100	100	100	100%
85	Muse Spark 1.1 (Reasoning, Medium)	$0.0078	9.2s	100%	100	100	100	100	100	100	100	100%
135	Z.AI GLM 5.1	$0.011	57.2s	100%	100	100	100	100	100	100	100	100%
158	Qwen3.6 Max Preview	$0.030	1.5m	100%	100	100	100	100	100	100	100	100%
137	GPT-5.5 (Reasoning)	$0.022	8.7s	100%	100	100	100	100	100	100	100	100%
144	Claude Sonnet 4.6 (Reasoning)	$0.023	13.0s	100%	100	100	100	100	100	100	100	100%
99	Z.AI GLM 5.2 (Reasoning, High)	$0.0077	20.2s	100%	100	100	100	100	100	100	100	100%
132	Gemini 3.5 Flash (Reasoning)	$0.021	8.6s	100%	100	100	100	100	100	100	100	100%
89	Z.AI GLM 5 Turbo	$0.0069	16.0s	100%	100	100	100	100	100	100	100	100%
133	MoonshotAI: Kimi K3 (Reasoning, High)	$0.017	29.1s	100%	100	100	100	100	100	100	100	100%
150	MoonshotAI: Kimi K2.6	$0.014	1.2m	100%	100	100	100	100	100	100	100	100%
140	Claude Opus 4.7 (Reasoning)	$0.024	4.1s	100%	100	100	100	100	100	100	100	100%
130	GPT-5.5 (Reasoning, Low)	$0.020	6.7s	100%	100	100	100	100	100	100	100	100%
82	GPT-5.6 Terra (Reasoning)	$0.0091	2.7s	100%	100	100	100	100	100	100	100	100%
145	Claude Opus 4.8 (Reasoning)	$0.024	8.4s	100%	100	100	100	100	100	100	100	100%
122	Claude Opus 4.6	$0.017	5.5s	100%	100	100	100	100	100	100	100	100%
114	MoonshotAI: Kimi K3 (Reasoning, Low)	$0.012	20.9s	100%	100	100	100	100	100	100	100	100%
147	Claude Opus 5 (Reasoning, Low)	$0.026	9.1s	100%	100	100	100	100	100	100	100	100%
143	Claude Opus 4.8 (Reasoning, Low)	$0.024	7.6s	100%	100	100	100	100	100	100	100	100%
155	GPT-5	$0.031	47.1s	100%	100	100	100	100	100	100	100	100%
61	Muse Spark 1.1 (Reasoning, Minimal)	$0.0055	6.1s	100%	100	100	100	100	100	100	100	100%
81	GPT-5 Mini	$0.0043	25.1s	100%	100	100	100	100	100	100	100	100%
146	Claude Opus 5 (Reasoning)	$0.026	8.2s	100%	100	100	100	100	100	100	100	100%
138	Qwen 3.5 397B A17B	$0.0040	1.6m	100%	100	100	100	100	100	100	100	100%
111	Grok 4.3 (Reasoning)	$0.0083	33.2s	100%	100	100	100	100	100	100	100	100%
102	Grok 4.5 (Reasoning, Low)	$0.0092	13.4s	100%	100	100	100	100	100	100	100	100%
91	GPT-5.4 (Reasoning, Low)	$0.0092	5.2s	100%	100	100	100	100	100	100	100	100%
75	Grok 4.20 (Reasoning)	$0.0051	16.5s	100%	100	100	100	100	100	100	100	100%
149	MoonshotAI: Kimi K2.5	$0.0082	1.6m	100%	100	100	100	100	100	100	100	100%
152	Thinking Machines Inkling (Reasoning)	$0.0099	1.6m	100%	100	100	100	100	100	100	100	100%
125	GPT-5.6 Sol	$0.018	4.9s	100%	100	100	100	100	100	100	100	100%
110	GPT-5.1	$0.012	13.4s	100%	100	100	100	100	100	100	100	100%
93	Claude Sonnet 4.6	$0.010	4.7s	100%	100	100	100	100	100	100	100	100%
47	GPT-5.6 Luna (Reasoning)	$0.0041	3.7s	100%	100	100	100	100	100	100	100	100%
71	MiniMax M3	$0.0019	29.5s	100%	100	100	100	100	100	100	100	100%
101	Claude Sonnet 5 (Reasoning)	$0.010	9.1s	100%	100	100	100	100	100	100	100	100%
127	Qwen 3.5 122B	$0.013	29.5s	100%	100	100	100	100	100	100	100	100%
105	Claude Sonnet 5 (Reasoning, Low)	$0.011	9.6s	100%	100	100	100	100	100	100	100	100%
115	Qwen 3.5 27B	$0.0092	37.2s	100%	100	100	100	100	100	100	100	100%
142	Claude Opus 5	$0.024	7.7s	100%	100	100	100	100	100	100	100	100%
86	Gemini 3 Flash (Preview, Reasoning)	$0.0076	12.0s	100%	100	100	100	100	100	100	100	100%
141	Claude Opus 4.7	$0.024	4.4s	100%	100	100	100	100	100	100	100	100%
49	GPT-5.4 Mini (Reasoning)	$0.0038	5.4s	100%	100	100	100	100	100	100	100	100%
131	Qwen 3.5 Plus (2026-04-20)	$0.0090	58.9s	100%	100	100	100	100	100	100	100	100%
113	Gemma 4 31B (Reasoning)	$0.0007	1.2m	100%	100	100	100	100	100	100	100	100%
120	Claude Opus 4.5	$0.017	5.3s	100%	100	100	100	100	100	100	100	100%
129	Z.AI GLM 5	$0.0082	57.8s	100%	100	100	100	100	100	100	100	100%
95	ByteDance Seed 1.6	$0.0033	37.9s	100%	100	100	100	100	100	100	100	100%
79	GPT-5.2	$0.0081	5.1s	100%	100	100	100	100	100	100	100	100%
123	GPT-5.5	$0.018	4.5s	100%	100	100	100	100	100	100	100	100%
78	Qwen 3.6 Flash	$0.0056	16.1s	100%	100	100	100	100	100	100	100	100%
88	DeepSeek V4 Pro (Reasoning)	$0.0015	41.9s	100%	100	100	100	100	100	100	100	100%
92	Gemma 4 26B (Reasoning)	$0.0012	47.6s	100%	100	100	100	100	100	100	100	100%
139	Aion 3.0	$0.015	46.0s	100%	100	100	100	100	100	100	100	100%
151	o4 Mini High	$0.022	36.3s	100%	100	100	100	100	100	100	100	100%
128	Gemini 2.5 Pro	$0.017	12.6s	100%	100	100	100	100	100	100	100	100%
134	Qwen 3.6 27B	$0.012	50.7s	100%	100	100	100	100	100	100	100	100%
90	GPT-5.6 Terra	$0.0091	6.1s	100%	100	100	100	100	100	100	100	100%
153	DeepSeek V4 Flash (Reasoning)	$0.0005	2.7m	100%	100	100	100	100	100	100	100	100%
98	Z.AI GLM 4.7	$0.0049	33.1s	100%	100	100	100	100	100	100	100	100%
84	Qwen 3.6 35B	$0.0045	25.2s	100%	100	100	100	100	100	100	100	100%
109	Z.AI GLM 4.6	$0.0045	47.8s	100%	100	100	100	100	100	100	100	100%
96	Claude Sonnet 4	$0.010	5.5s	100%	100	100	100	100	100	100	100	100%
94	Claude Sonnet 4.5	$0.010	4.8s	100%	100	100	100	100	100	100	100	100%
97	Claude Sonnet 5	$0.0095	9.2s	100%	100	100	100	100	100	100	100	100%
159	Claude Opus 4	$0.051	7.9s	100%	100	100	100	100	100	100	100	100%
112	Qwen 3.5 35B	$0.0095	27.8s	100%	100	100	100	100	100	100	100	100%
58	GPT-4.1	$0.0053	4.4s	100%	100	100	100	100	100	100	100	100%
100	MiniMax M2.5	$0.0031	42.3s	100%	100	100	100	100	100	100	100	100%
52	Gemini 3.6 Flash (Reasoning, Minimal)	$0.0047	2.2s	100%	100	100	100	100	100	100	100	100%
106	Aion 2.0	$0.0037	43.5s	100%	100	100	100	100	100	100	100	100%
126	o4 Mini	$0.015	23.3s	100%	100	100	100	100	100	100	100	100%
34	Qwen 3.5 Plus (2026-02-15)	$0.0015	7.1s	100%	100	100	100	100	100	100	100	100%
66	Xiaomi MIMO v2.5 Pro	$0.0040	16.8s	100%	100	100	100	100	100	100	100	100%
10	Gemini 3.1 Flash Lite (Reasoning)	$0.0009	1.7s	100%	100	100	100	100	100	100	100	100%
57	Gemini 3.5 Flash (Reasoning, Minimal)	$0.0055	2.5s	100%	100	100	100	100	100	100	100	100%
117	ByteDance Seed 2.0 Mini	$0.0013	1.3m	100%	100	100	100	100	100	100	100	100%
77	Qwen 3.5 Flash	$0.0021	32.1s	100%	100	100	100	100	100	100	100	100%
28	Gemini 3 Flash (Preview)	$0.0018	3.2s	100%	100	100	100	100	100	100	100	100%
9	Gemini 3.1 Flash Lite (Preview)	$0.0009	1.6s	100%	100	100	100	100	100	100	100	100%
39	Gemma 4 31B	$0.0003	15.9s	100%	100	100	100	100	100	100	100	100%
12	Gemini 3.1 Flash Lite	$0.0009	1.9s	100%	100	100	100	100	100	100	100	100%
44	GPT-5.6 Luna	$0.0036	2.3s	100%	100	100	100	100	100	100	100	100%
83	Z.AI GLM 4.5	$0.0030	31.6s	100%	100	100	100	100	100	100	100	100%
37	Gemma 4 26B	$0.0003	15.4s	100%	100	100	100	100	100	100	100	100%
107	GPT-OSS 120B	$0.0007	57.9s	100%	100	100	100	100	100	100	100	100%
15	Gemini 3.5 Flash Lite (Reasoning, Minimal)	$0.0014	1.4s	100%	100	100	100	100	100	100	100	100%
16	Gemini 3.5 Flash Lite (Reasoning)	$0.0014	1.4s	100%	100	100	100	100	100	100	100	100%
87	GPT-5.4	$0.0091	5.3s	100%	100	100	100	100	100	100	100	100%
30	Mistral Large 3	$0.0011	7.4s	100%	100	100	100	100	100	100	100	100%
104	ByteDance Seed 2.0 Lite	$0.0037	40.4s	100%	100	100	100	100	100	100	100	100%
46	Gemini 2.5 Flash (Reasoning)	$0.0037	5.6s	100%	100	100	100	100	100	100	100	100%
45	DeepSeek-V2 Chat	$0.0008	16.9s	100%	100	100	100	100	100	100	100	100%
119	Qwen 3.5 9B	$0.0009	1.3m	100%	100	100	100	100	100	100	100	100%
56	Xiaomi MIMO v2.5	$0.0033	12.7s	100%	100	100	100	100	100	100	100	100%
118	Aion 3.0 Mini	$0.0040	1.1m	100%	100	100	100	100	100	100	100	100%
41	GPT-5.4 Mini (Reasoning, Low)	$0.0030	3.8s	100%	100	100	100	100	100	100	100	100%
43	Claude Haiku 4.5	$0.0034	3.0s	100%	100	100	100	100	100	100	100	100%
55	Gemini 2.5 Flash Lite (Reasoning)	$0.0010	23.4s	100%	100	100	100	100	100	100	100	100%
40	DeepSeek V3 (2024-12-26)	$0.0008	14.3s	100%	100	100	100	100	100	100	100	100%
68	DeepSeek V3.1	$0.0007	32.9s	100%	100	100	100	100	100	100	100	100%
62	DeepSeek V3.2	$0.0004	30.7s	100%	100	100	100	100	100	100	100	100%
65	GPT-4o, Aug. 6th (temp=0)	$0.0066	2.3s	100%	100	100	100	100	100	100	100	100%
36	DeepSeek V4 Pro	$0.0009	11.9s	100%	100	100	100	100	100	100	100	100%
21	DeepSeek V4 Flash	$0.0002	8.2s	100%	100	100	100	100	100	100	100	100%
23	Inception Mercury 2	$0.0017	2.3s	100%	100	100	100	100	100	100	100	100%
69	Nemotron 3 Super	$0.0000	37.7s	100%	100	100	100	100	100	100	100	100%
54	Mistral Large 2	$0.0043	7.5s	100%	100	100	100	100	100	100	100	100%
24	GPT-4.1 Mini	$0.0011	6.0s	100%	100	100	100	100	100	100	100	100%
64	GPT-4o, Aug. 6th (temp=1)	$0.0066	2.2s	100%	100	100	100	100	100	100	100	100%
31	Grok 4.20	$0.0020	4.0s	100%	100	100	100	100	100	100	100	100%
60	Hermes 3 405B	$0.0012	24.7s	100%	100	100	100	100	100	100	100	100%
63	Z.AI GLM 4.5 Air	$0.0015	26.3s	100%	100	100	100	100	100	100	100	100%
20	Gemini 2.5 Flash	$0.0014	2.0s	100%	100	100	100	100	100	100	100	100%
35	GPT-5.4 Mini	$0.0027	2.4s	100%	100	100	100	100	100	100	100	100%
103	GPT-5 Nano	$0.0021	47.8s	100%	100	100	100	100	100	100	100	100%
18	GPT-5.4 Nano (Reasoning)	$0.0009	4.3s	100%	100	100	100	100	100	100	100	100%
17	Laguna S 2.1	$0.0002	7.8s	100%	100	100	100	100	100	100	100	100%
74	DeepSeek V3 (2025-03-24)	$0.0006	37.3s	100%	100	100	100	100	100	100	100	100%
3	Gemini 2.5 Flash Lite	$0.0003	1.5s	100%	100	100	100	100	100	100	100	100%
51	Qwen 3 32B	$0.0007	21.4s	100%	100	100	100	100	100	100	100	100%
53	Writer: Palmyra X5	$0.0035	11.0s	100%	100	100	100	100	100	100	100	100%
42	Qwen3 235B A22B Instruct 2507	$0.0004	16.9s	100%	100	100	100	100	100	100	100	100%
25	GPT-4o Mini (temp=1)	$0.0004	9.5s	100%	100	100	100	100	100	100	100	100%
13	GPT-5.4 Nano (Reasoning, Low)	$0.0007	2.8s	100%	100	100	100	100	100	100	100	100%
50	Llama 3.1 70B	$0.0005	22.0s	100%	100	100	100	100	100	100	100	100%
8	Mistral Small 3.2 24B	$0.0002	4.5s	100%	100	100	100	100	100	100	100	100%
29	GPT-4o Mini (temp=0)	$0.0004	10.1s	100%	100	100	100	100	100	100	100	100%
32	Mistral Medium 3.1	$0.0012	8.0s	100%	100	100	100	100	100	100	100	100%
19	Gemma 3 12B	$0.0001	8.5s	100%	100	100	100	100	100	100	100	100%
33	Gemma 3 27B	$0.0002	13.2s	100%	100	100	100	100	100	100	100	100%
22	Laguna XS 2.1	$0.0002	8.7s	100%	100	100	100	100	100	100	100	100%
6	Mistral Small 4	$0.0004	2.8s	100%	100	100	100	100	100	100	100	100%
48	Thinking Machines Inkling	$0.0027	10.7s	100%	100	100	100	100	100	100	100	100%
26	Qwen 2.5 72B	$0.0003	10.4s	100%	100	100	100	100	100	100	100	100%
38	Cydonia 24B V4.1	$0.0004	14.9s	100%	100	100	100	100	100	100	100	100%
14	GPT-5.4 Nano	$0.0007	3.0s	100%	100	100	100	100	100	100	100	100%
70	WizardLM 2 8x22b	$0.0007	34.7s	100%	100	100	100	100	100	100	100	100%
27	ByteDance Seed 1.6 Flash	$0.0005	9.3s	100%	100	100	100	100	100	100	100	100%
7	Ministral 3 14B	$0.0002	3.8s	100%	100	100	100	100	100	100	100	100%
5	Ministral 3 8B	$0.0002	3.1s	100%	100	100	100	100	100	100	100	100%
80	Cohere Command R+ (Aug. 2024)	$0.0069	11.1s	100%	100	100	100	100	100	100	100	100%
11	Gemma 3 4B	$0.0001	5.8s	100%	100	100	100	100	100	100	100	100%
2	Ministral 3 3B	$0.0001	1.9s	100%	100	100	100	100	100	100	100	100%
4	Ministral 8B	$0.0001	3.2s	100%	100	100	100	100	100	100	100	100%
1	Ministral 3B	$0.0000	1.9s	100%	100	100	100	100	100	100	100	100%
76	Mistral Small 4 (Reasoning)	$0.0019	16.6s	96%	100	100	100	100	100	96	96	99%
59	Grok 4.3	$0.0021	4.6s	96%	100	100	100	100	100	96	96	99%
72	Arcee AI: Trinity Mini	$0.0002	5.2s	92%	100	100	98	96	96	96	96	97%
136	Z.AI GLM 4.7 Flash	$0.0011	40.6s	82%	100	100	100	100	100	100	74	96%
157	MiniMax M2.7	$0.0085	1.7m	80%	100	100	100	100	100	100	71	96%
67	GPT-4.1 Nano	$0.0003	3.6s	96%	96	96	96	96	96	96	96	96%
73	Mistral NeMO	$0.0002	2.3s	94%	96	96	96	96	96	96	93	95%
160	Gemini 3.1 Pro (Preview)	$0.031	30.3s	72%	100	100	100	100	100	100	60	94%
156	Nemotron 3 Nano	$0.0018	1.8m	74%	100	100	100	100	100	98	63	94%
161	Hermes 3 70B	$0.0030	4.1m	4%	100	100	100	54	0	0	0	51%
99.49%