Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Text Replacement

Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.

Text Editing

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
GPT-5.6 Sol (Reasoning)	100%
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.6 Flash (Reasoning)	100%
Qwen3.7 Max	100%
Grok 4.5 (Reasoning, High)	100%
Gemini 3.1 Pro (Preview)	100%
Qwen3.6 Max Preview	100%
Claude Sonnet 4.6 (Reasoning)	100%
MoonshotAI: Kimi K3 (Reasoning, High)	100%
MoonshotAI: Kimi K2.6	100%
GPT-5.5 (Reasoning, Low)	100%
GPT-5.6 Terra (Reasoning)	100%
Claude Opus 4.8 (Reasoning)	100%
Claude Opus 4.6	100%
MoonshotAI: Kimi K3 (Reasoning, Low)	100%
Claude Opus 5 (Reasoning, Low)	100%
Claude Opus 4.8 (Reasoning, Low)	100%
GPT-5	100%
Muse Spark 1.1 (Reasoning, Minimal)	100%
Claude Opus 5 (Reasoning)	100%

	Score	Cost	Time
Gemini 3.1 Flash Lite	100%	$0.0010	5.9s
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0010	2.0s
GPT-5.4 Nano (Reasoning, Low)	94%	$0.0008	3.3s
Gemma 4 26B	100%	$0.0003	12.6s
Gemini 2.5 Flash	100%	$0.0017	2.4s
Z.AI GLM 4.5 Air	99%	$0.0016	43.4s
Mistral Medium 3.1	100%	$0.0014	6.9s
GPT-4.1 Mini	100%	$0.0012	6.5s
Gemini 3.5 Flash Lite (Reasoning, Minimal)	100%	$0.0017	1.8s
Gemini 3.5 Flash Lite (Reasoning)	99%	$0.0017	1.5s
Inception Mercury 2	97%	$0.0014	2.0s
Qwen 2.5 72B	89%	$0.0003	11.6s
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s
DeepSeek V4 Flash (Reasoning)	100%	$0.0004	16.6s
ByteDance Seed 1.6 Flash	95%	$0.0007	12.0s
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s
Grok 4.20	94%	$0.0021	4.5s
Gemma 4 31B	100%	$0.0004	19.1s
DeepSeek V4 Pro	100%	$0.0020	32.0s

	Score	Cost	Speed	Stability
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s	100%
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0010	2.0s	100%
Gemini 2.5 Flash	100%	$0.0017	2.4s	100%
Gemini 3.1 Flash Lite	100%	$0.0010	5.9s	100%
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s	100%
Gemini 3.5 Flash Lite (Reasoning)	99%	$0.0017	1.5s	99%
GPT-4.1 Mini	100%	$0.0012	6.5s	100%
Gemini 3.5 Flash Lite (Reasoning, Minimal)	100%	$0.0017	1.8s	99%
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s	100%
Mistral Medium 3.1	100%	$0.0014	6.9s	100%
GPT-5.6 Luna	100%	$0.0041	2.7s	100%
Claude Haiku 4.5	100%	$0.0041	3.1s	100%
Gemma 4 26B	100%	$0.0003	12.6s	100%
GPT-5.6 Luna (Reasoning)	100%	$0.0045	3.1s	100%
Gemini 3.6 Flash (Reasoning, Minimal)	100%	$0.0054	2.6s	100%
DeepSeek V4 Flash (Reasoning)	100%	$0.0004	16.6s	100%
Muse Spark 1.1 (Reasoning, Minimal)	100%	$0.0051	5.6s	100%
Gemini 3.5 Flash (Reasoning, Minimal)	100%	$0.0063	2.7s	100%
GPT-4.1	100%	$0.0059	4.6s	100%
Gemma 4 31B	100%	$0.0004	19.1s	100%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	Total
71	GPT-5.6 Sol (Reasoning)	$0.023	6.4s	100%	100	100	100	100	100	100	100	100%
67	Claude Opus 4.6 (Reasoning)	$0.021	7.3s	100%	100	100	100	100	100	100	100	100%
66	Gemini 3.6 Flash (Reasoning)	$0.020	9.8s	100%	100	100	100	100	100	100	100	100%
120	Qwen3.7 Max	$0.029	50.7s	100%	100	100	100	100	100	100	100	100%
55	Grok 4.5 (Reasoning, High)	$0.013	18.3s	100%	100	100	100	100	100	100	100	100%
116	Gemini 3.1 Pro (Preview)	$0.032	31.1s	100%	100	100	100	100	100	100	100	100%
142	Qwen3.6 Max Preview	$0.030	1.6m	100%	100	100	100	100	100	100	100	100%
43	Claude Sonnet 4.6 (Reasoning)	$0.013	6.1s	100%	100	100	100	100	100	100	100	100%
83	MoonshotAI: Kimi K3 (Reasoning, High)	$0.018	32.2s	100%	100	100	100	100	100	100	100	100%
108	MoonshotAI: Kimi K2.6	$0.012	1.2m	100%	100	100	100	100	100	100	100	100%
61	GPT-5.5 (Reasoning, Low)	$0.021	5.6s	100%	100	100	100	100	100	100	100	100%
28	GPT-5.6 Terra (Reasoning)	$0.011	3.8s	100%	100	100	100	100	100	100	100	100%
88	Claude Opus 4.8 (Reasoning)	$0.029	9.5s	100%	100	100	100	100	100	100	100	100%
58	Claude Opus 4.6	$0.020	6.4s	100%	100	100	100	100	100	100	100	100%
53	MoonshotAI: Kimi K3 (Reasoning, Low)	$0.011	20.9s	100%	100	100	100	100	100	100	100	100%
96	Claude Opus 5 (Reasoning, Low)	$0.031	10.7s	100%	100	100	100	100	100	100	100	100%
85	Claude Opus 4.8 (Reasoning, Low)	$0.028	9.2s	100%	100	100	100	100	100	100	100	100%
122	GPT-5	$0.034	43.1s	100%	100	100	100	100	100	100	100	100%
17	Muse Spark 1.1 (Reasoning, Minimal)	$0.0051	5.6s	100%	100	100	100	100	100	100	100	100%
93	Claude Opus 5 (Reasoning)	$0.030	10.4s	100%	100	100	100	100	100	100	100	100%
42	Grok 4.5 (Reasoning, Low)	$0.010	12.6s	100%	100	100	100	100	100	100	100	100%
31	GPT-5.4 (Reasoning, Low)	$0.011	6.5s	100%	100	100	100	100	100	100	100	100%
30	Grok 4.20 (Reasoning)	$0.0060	18.4s	100%	100	100	100	100	100	100	100	100%
119	MoonshotAI: Kimi K2.5	$0.0075	1.6m	100%	100	100	100	100	100	100	100	100%
81	Thinking Machines Inkling (Reasoning)	$0.0090	53.8s	100%	100	100	100	100	100	100	100	100%
51	GPT-5.1	$0.013	13.8s	100%	100	100	100	100	100	100	100	100%
32	Claude Sonnet 4.6	$0.012	5.1s	100%	100	100	100	100	100	100	100	100%
14	GPT-5.6 Luna (Reasoning)	$0.0045	3.1s	100%	100	100	100	100	100	100	100	100%
47	Claude Sonnet 5 (Reasoning)	$0.012	11.6s	100%	100	100	100	100	100	100	100	100%
41	Claude Sonnet 5 (Reasoning, Low)	$0.011	9.7s	100%	100	100	100	100	100	100	100	100%
84	Claude Opus 5	$0.028	8.4s	100%	100	100	100	100	100	100	100	100%
24	Gemini 3 Flash (Preview, Reasoning)	$0.0065	10.3s	100%	100	100	100	100	100	100	100	100%
57	Claude Opus 4.5	$0.020	5.7s	100%	100	100	100	100	100	100	100	100%
52	Z.AI GLM 5	$0.0044	34.8s	100%	100	100	100	100	100	100	100	100%
49	ByteDance Seed 1.6	$0.0031	34.1s	100%	100	100	100	100	100	100	100	100%
56	GPT-5.5	$0.021	5.2s	100%	100	100	100	100	100	100	100	100%
101	DeepSeek V4 Pro (Reasoning)	$0.0063	1.3m	100%	100	100	100	100	100	100	100	100%
114	Gemma 4 26B (Reasoning)	$0.0017	1.8m	100%	100	100	100	100	100	100	100	100%
82	Aion 3.0	$0.014	41.8s	100%	100	100	100	100	100	100	100	100%
99	Gemini 2.5 Pro	$0.028	21.0s	100%	100	100	100	100	100	100	100	100%
27	GPT-5.6 Terra	$0.010	3.7s	100%	100	100	100	100	100	100	100	100%
16	DeepSeek V4 Flash (Reasoning)	$0.0004	16.6s	100%	100	100	100	100	100	100	100	100%
72	Z.AI GLM 4.7	$0.0055	50.8s	100%	100	100	100	100	100	100	100	100%
48	Z.AI GLM 4.6	$0.0051	27.9s	100%	100	100	100	100	100	100	100	100%
36	Claude Sonnet 4	$0.012	6.4s	100%	100	100	100	100	100	100	100	100%
33	Claude Sonnet 4.5	$0.012	5.2s	100%	100	100	100	100	100	100	100	100%
38	Claude Sonnet 5	$0.011	9.8s	100%	100	100	100	100	100	100	100	100%
136	Claude Opus 4	$0.061	9.1s	100%	100	100	100	100	100	100	100	100%
19	GPT-4.1	$0.0059	4.6s	100%	100	100	100	100	100	100	100	100%
15	Gemini 3.6 Flash (Reasoning, Minimal)	$0.0054	2.6s	100%	100	100	100	100	100	100	100	100%
65	Aion 2.0	$0.0040	49.3s	100%	100	100	100	100	100	100	100	100%
25	MiniMax M2.7	$0.0026	22.1s	100%	100	100	100	100	100	100	100	100%
9	Qwen 3.5 Plus (2026-02-15)	$0.0017	7.6s	100%	100	100	100	100	100	100	100	100%
2	Gemini 3.1 Flash Lite (Reasoning)	$0.0010	2.0s	100%	100	100	100	100	100	100	100	100%
18	Gemini 3.5 Flash (Reasoning, Minimal)	$0.0063	2.7s	100%	100	100	100	100	100	100	100	100%
5	Gemini 3 Flash (Preview)	$0.0021	3.5s	100%	100	100	100	100	100	100	100	100%
1	Gemini 3.1 Flash Lite (Preview)	$0.0010	1.9s	100%	100	100	100	100	100	100	100	100%
20	Gemma 4 31B	$0.0004	19.1s	100%	100	100	100	100	100	100	100	100%
4	Gemini 3.1 Flash Lite	$0.0010	5.9s	100%	100	100	100	100	100	100	100	100%
11	GPT-5.6 Luna	$0.0041	2.7s	100%	100	100	100	100	100	100	100	100%
45	Z.AI GLM 4.5	$0.0037	30.4s	100%	100	100	100	100	100	100	100	100%
13	Gemma 4 26B	$0.0003	12.6s	100%	100	100	100	100	100	100	100	100%
60	ByteDance Seed 2.0 Lite	$0.0040	46.7s	100%	100	100	100	100	100	100	100	100%
63	Aion 3.0 Mini	$0.0029	50.8s	100%	100	100	100	100	100	100	100	100%
12	Claude Haiku 4.5	$0.0041	3.1s	100%	100	100	100	100	100	100	100	100%
21	GPT-4o, Aug. 6th (temp=0)	$0.0074	2.7s	100%	100	100	100	100	100	100	100	100%
39	DeepSeek V4 Pro	$0.0020	32.0s	100%	100	100	100	100	100	100	100	100%
7	GPT-4.1 Mini	$0.0012	6.5s	100%	100	100	100	100	100	100	100	100%
26	Hermes 3 405B	$0.0013	25.6s	100%	100	100	100	100	100	100	100	100%
3	Gemini 2.5 Flash	$0.0017	2.4s	100%	100	100	100	100	100	100	100	100%
37	Qwen 3 32B	$0.0006	35.1s	100%	100	100	100	100	100	100	100	100%
89	Z.AI GLM 5.1	$0.011	51.9s	100%	100	100	100	100	100	100	99	100%
76	GPT-5.5 (Reasoning)	$0.025	7.9s	100%	100	100	100	100	100	100	99	100%
79	Gemini 3.5 Flash (Reasoning)	$0.025	10.4s	100%	100	100	100	100	100	100	99	100%
59	GPT-5.6 Sol	$0.021	4.8s	100%	100	100	100	100	100	100	99	100%
73	o4 Mini High	$0.017	25.4s	100%	100	100	100	100	100	100	99	100%
10	Mistral Medium 3.1	$0.0014	6.9s	100%	100	100	100	100	100	100	99	100%
110	Qwen 3.5 122B	$0.021	49.1s	99%	100	100	100	100	100	99	99	100%
103	Qwen 3.6 27B	$0.013	1.0m	99%	100	100	100	100	100	99	99	100%
54	o4 Mini	$0.012	18.0s	99%	100	100	100	100	100	99	99	100%
29	Muse Spark 1.1 (Reasoning, Medium)	$0.0082	8.8s	99%	100	100	100	100	99	99	99	100%
64	Grok 4.3 (Reasoning)	$0.0074	38.1s	99%	100	100	100	100	99	99	99	100%
40	MiniMax M3	$0.0020	30.3s	99%	100	100	100	100	99	99	99	100%
22	GPT-4o, Aug. 6th (temp=1)	$0.0074	3.5s	99%	100	100	100	100	99	99	99	100%
77	Gemma 4 31B (Reasoning)	$0.0007	1.1m	99%	100	100	100	99	99	99	99	100%
50	Qwen 3.6 35B	$0.0048	27.0s	99%	100	100	100	99	99	99	99	100%
8	Gemini 3.5 Flash Lite (Reasoning, Minimal)	$0.0017	1.8s	99%	100	100	100	99	99	99	99	100%
90	Claude Opus 4.7 (Reasoning)	$0.028	4.6s	98%	100	100	100	100	100	100	97	100%
46	Qwen 3.6 Flash	$0.0062	18.8s	98%	100	100	99	99	99	99	99	99%
44	GPT-5.4	$0.010	6.0s	98%	100	100	100	100	99	99	97	99%
95	Qwen 3.5 27B	$0.012	53.2s	99%	99	99	99	99	99	99	99	99%
35	GPT-OSS 120B	$0.0007	29.7s	99%	99	99	99	99	99	99	99	99%
6	Gemini 3.5 Flash Lite (Reasoning)	$0.0017	1.5s	99%	99	99	99	99	99	99	99	99%
62	Z.AI GLM 4.5 Air	$0.0016	43.4s	97%	100	100	100	100	99	99	95	99%
34	Gemini 2.5 Flash (Reasoning)	$0.0055	8.3s	96%	99	99	99	99	99	99	95	99%
23	Inception Mercury 2	$0.0014	2.0s	95%	99	99	99	99	95	95	95	97%
112	MiniMax M2.5	$0.0017	1.1m	89%	100	99	99	99	99	97	85	97%
102	Claude Opus 4.7	$0.028	5.1s	94%	100	97	97	97	96	96	96	97%
94	Z.AI GLM 5.2 (Reasoning, High)	$0.0048	15.4s	82%	100	100	100	100	100	100	74	96%
104	Z.AI GLM 5 Turbo	$0.0086	18.8s	82%	100	100	100	100	100	100	74	96%
100	Llama 3.1 70B	$0.0005	34.1s	82%	100	100	100	100	100	100	74	96%
91	Thinking Machines Inkling	$0.0030	18.3s	82%	100	100	100	100	100	100	74	96%
111	GPT-5.4 (Reasoning)	$0.014	11.6s	82%	100	100	100	100	100	99	74	96%
97	GPT-5.2	$0.010	7.0s	82%	100	100	100	100	99	99	74	96%
98	Xiaomi MIMO v2.5	$0.0052	20.7s	82%	100	100	100	100	99	99	74	96%
131	Qwen 3.5 Plus (2026-04-20)	$0.011	1.1m	81%	100	100	100	99	99	99	74	96%
148	Qwen 3.5 9B	$0.0014	2.1m	81%	100	100	100	99	99	99	74	96%
106	GPT-5 Mini	$0.0047	28.0s	81%	100	100	100	99	99	97	74	96%
80	ByteDance Seed 1.6 Flash	$0.0007	12.0s	81%	99	99	99	99	99	99	72	95%
70	Gemma 3 27B	$0.0002	17.7s	89%	98	98	93	93	93	93	93	94%
78	GPT-5.4 Nano (Reasoning, Low)	$0.0008	3.3s	80%	100	99	99	99	99	86	74	94%
86	Grok 4.20	$0.0021	4.5s	80%	100	100	100	100	100	83	74	94%
92	Laguna S 2.1	$0.0002	16.1s	81%	100	100	100	93	91	85	85	93%
157	Qwen 3.5 397B A17B	$0.010	2.5m	76%	100	100	100	100	100	74	74	92%
109	GPT-5.4 Mini (Reasoning)	$0.0061	5.8s	76%	100	100	100	100	100	74	74	92%
105	Laguna XS 2.1	$0.0003	14.5s	76%	100	100	100	99	99	75	73	92%
69	Qwen3 235B A22B Instruct 2507	$0.0003	9.7s	91%	91	91	91	91	91	91	91	91%
68	Arcee AI: Trinity Mini	$0.0002	7.4s	90%	91	91	91	91	91	90	90	91%
107	GPT-5.4 Nano	$0.0008	3.7s	75%	99	99	99	93	83	80	75	90%
152	Qwen 3.5 35B	$0.017	53.6s	70%	100	100	99	90	90	73	73	89%
74	Ministral 3 3B	$0.0001	2.2s	87%	91	91	91	91	88	85	85	89%
126	Nemotron 3 Super	$0.0000	47.0s	74%	99	99	95	94	90	73	73	89%
115	GPT-5.4 Mini (Reasoning, Low)	$0.0035	2.9s	74%	100	100	100	100	74	74	74	89%
117	Qwen 2.5 72B	$0.0003	11.6s	74%	100	100	100	100	74	74	74	89%
75	Gemma 3 4B	$0.0001	6.3s	89%	89	89	89	89	89	89	89	89%
139	GPT-5 Nano	$0.0028	58.3s	73%	100	100	99	97	75	75	73	89%
113	Writer: Palmyra X5	$0.0039	12.4s	80%	91	91	91	91	91	91	74	88%
87	Ministral 3B	$0.0001	2.2s	85%	88	88	88	88	85	85	85	87%
141	Mistral Small 4 (Reasoning)	$0.0016	14.1s	55%	100	100	99	99	95	73	37	86%
125	Cydonia 24B V4.1	$0.0004	13.1s	65%	100	97	91	80	80	78	74	86%
143	Xiaomi MIMO v2.5 Pro	$0.0026	11.4s	55%	100	100	100	100	100	50	50	86%
123	GPT-5.4 Nano (Reasoning)	$0.0010	3.7s	68%	100	99	92	88	74	74	73	86%
145	DeepSeek-V2 Chat	$0.0009	16.4s	54%	100	100	100	74	74	74	74	85%
153	Qwen 3.5 Flash	$0.0037	57.2s	64%	100	99	91	83	73	73	73	85%
146	DeepSeek V3 (2025-03-24)	$0.0007	21.4s	56%	100	100	91	74	74	74	70	83%
156	Z.AI GLM 4.7 Flash	$0.0018	1.2m	55%	100	99	92	74	74	72	72	83%
147	DeepSeek V3 (2024-12-26)	$0.0008	16.6s	56%	100	100	74	74	74	74	74	81%
150	Hermes 3 70B	$0.0004	22.6s	56%	100	100	74	74	74	74	74	81%
121	GPT-5.4 Mini	$0.0031	2.3s	78%	83	83	83	83	83	80	74	81%
118	Mistral Small 3.2 24B	$0.0002	6.0s	80%	80	80	80	80	80	80	80	80%
124	Mistral Small 4	$0.0004	3.5s	72%	85	85	78	78	78	78	74	79%
127	Ministral 3 14B	$0.0003	4.4s	69%	88	88	78	78	76	74	74	79%
154	WizardLM 2 8x22b	$0.0008	42.8s	60%	100	74	74	74	74	74	74	77%
151	Gemini 2.5 Flash Lite (Reasoning)	$0.0026	19.1s	60%	100	74	74	74	74	73	73	77%
161	Nemotron 3 Nano	$0.0018	1.7m	42%	99	99	95	87	74	39	33	75%
130	Ministral 3 8B	$0.0002	3.5s	71%	78	78	74	74	74	74	74	75%
129	Ministral 8B	$0.0001	4.0s	73%	75	74	74	74	74	74	74	74%
134	Mistral Large 3	$0.0012	8.4s	74%	74	74	74	74	74	74	74	74%
149	DeepSeek V3.2	$0.0004	38.8s	74%	74	74	74	74	74	74	74	74%
132	DeepSeek V4 Flash	$0.0002	7.9s	74%	74	74	74	74	74	74	74	74%
140	Mistral Large 2	$0.0048	8.3s	74%	74	74	74	74	74	74	74	74%
138	GPT-4o Mini (temp=1)	$0.0004	11.9s	74%	74	74	74	74	74	74	74	74%
135	GPT-4o Mini (temp=0)	$0.0004	10.7s	74%	74	74	74	74	74	74	74	74%
133	Gemma 3 12B	$0.0001	8.7s	74%	74	74	74	74	74	74	74	74%
128	Mistral NeMO	$0.0002	1.8s	74%	74	74	74	74	74	74	74	74%
137	Grok 4.3	$0.0023	5.8s	73%	74	74	74	74	74	74	73	74%
158	ByteDance Seed 2.0 Mini	$0.0018	1.8m	73%	74	74	74	74	74	73	72	73%
155	Cohere Command R+ (Aug. 2024)	$0.0077	41.5s	69%	74	74	74	74	74	74	65	72%
144	Gemini 2.5 Flash Lite	$0.0003	2.0s	67%	72	72	72	72	72	72	63	70%
160	DeepSeek V3.1	$0.0006	25.1s	38%	74	74	74	74	74	74	5	64%
159	GPT-4.1 Nano	$0.0003	4.5s	46%	63	51	51	51	51	51	50	52%
93.76%

Median	Evaluator	Top 3	Flop 3
100.0%	Dialogue content preserved	100GPT-5.4 Nano (Reasoning, Low) 100Claude Sonnet 5 (Reasoning) 100DeepSeek V4 Pro	84Nemotron 3 Nano 86DeepSeek V3.1 91Ministral 3B
99.6%	Gender swap transformation accuracy	100Hermes 3 405B 100Aion 2.0 100Grok 4.5 (Reasoning, Low)	9GPT-4.1 Nano 71Xiaomi MIMO v2.5 Pro 78Nemotron 3 Nano
100.0%	Mara pronouns preserved (coreference test)	100Grok 4.5 (Reasoning, High) 100Z.AI GLM 5.1 100Grok 4.20 (Reasoning)	0Mistral NeMO 0Mistral Large 2 0Gemma 3 12B
100.0%	Setting and description preserved	100GPT-5.5 (Reasoning, Low) 100Thinking Machines Inkling (Reasoning) 100GPT-5 Nano	83Nemotron 3 Nano 89DeepSeek V3.1 91Mistral Small 4 (Reasoning)

Text Replacement

Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)