Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Text Replacement

Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.

Text Editing

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Qwen3.7 Max	100%
Claude Opus 4.6 (Reasoning)	100%
Qwen3.6 Max Preview	100%
Gemini 3.1 Pro (Preview)	100%
Claude Sonnet 4.6 (Reasoning)	100%
GPT-5.5 (Reasoning, Low)	100%
GPT-5.1	100%
Claude Opus 4.6	100%
MoonshotAI: Kimi K2.6	100%
GPT-5	100%
Gemma 4 26B (Reasoning)	100%
Grok 4.20 (Beta, Reasoning)	100%
GPT-5.4 (Reasoning, Low)	100%
Grok 4.20 (Reasoning)	100%
Z.AI GLM 5	100%
Claude Sonnet 4.6	100%
MoonshotAI: Kimi K2.5	100%
ByteDance Seed 1.6	100%
Gemini 3 Flash (Preview, Reasoning)	100%
DeepSeek V4 Pro (Reasoning)	100%

	Score	Cost	Time
Mistral Small Creative	100%	$0.0002	3.2s
Gemini 3.1 Flash Lite	100%	$0.0010	5.9s
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0010	2.0s
Grok 4 Fast	100%	$0.0007	5.0s
GPT-5.4 Nano (Reasoning, Low)	94%	$0.0008	3.3s
Gemma 4 26B	100%	$0.0003	12.6s
Stealth: Hunter Alpha	100%	$0.0000	12.6s
Gemini 2.5 Flash	100%	$0.0017	2.4s
Z.AI GLM 4.5 Air	99%	$0.0016	43.4s
Grok 4.1 Fast	100%	$0.0007	10.2s
Mistral Medium 3.1	100%	$0.0014	6.9s
GPT-4.1 Mini	100%	$0.0012	6.5s
Inception Mercury 2	97%	$0.0014	2.0s
Qwen 2.5 72B	89%	$0.0003	11.6s
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s
Stealth: Healer Alpha	99%	$0.0000	13.9s
ByteDance Seed 1.6 Flash	95%	$0.0007	12.0s
DeepSeek V4 Flash (Reasoning)	100%	$0.0004	16.6s
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s

	Score	Cost	Speed	Stability
Mistral Small Creative	100%	$0.0002	3.2s	100%
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s	100%
Gemini 3.1 Flash Lite (Reasoning)	100%	$0.0010	2.0s	100%
Gemini 2.5 Flash	100%	$0.0017	2.4s	100%
Grok 4 Fast	100%	$0.0007	5.0s	100%
Gemini 3.1 Flash Lite	100%	$0.0010	5.9s	100%
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s	100%
GPT-4.1 Mini	100%	$0.0012	6.5s	100%
Mistral Medium 3.1	100%	$0.0014	6.9s	100%
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s	100%
Grok 4.1 Fast	100%	$0.0007	10.2s	100%
Stealth: Hunter Alpha	100%	$0.0000	12.6s	100%
Claude Haiku 4.5	100%	$0.0041	3.1s	100%
Gemma 4 26B	100%	$0.0003	12.6s	100%
DeepSeek V4 Flash (Reasoning)	100%	$0.0004	16.6s	100%
Gemini 3.5 Flash (Reasoning, Minimal)	100%	$0.0063	2.7s	100%
Stealth: Healer Alpha	99%	$0.0000	13.9s	99%
GPT-4.1	100%	$0.0059	4.6s	100%
Gemma 4 31B	100%	$0.0004	19.1s	100%
Llama 3.1 Nemotron 70B	100%	$0.0015	16.8s	100%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	Total
105	Qwen3.7 Max	$0.029	50.7s	100%	100	100	100	100	100	100	100	100%
62	Claude Opus 4.6 (Reasoning)	$0.021	7.3s	100%	100	100	100	100	100	100	100	100%
134	Qwen3.6 Max Preview	$0.030	1.6m	100%	100	100	100	100	100	100	100	100%
101	Gemini 3.1 Pro (Preview)	$0.032	31.1s	100%	100	100	100	100	100	100	100	100%
41	Claude Sonnet 4.6 (Reasoning)	$0.013	6.1s	100%	100	100	100	100	100	100	100	100%
56	GPT-5.5 (Reasoning, Low)	$0.021	5.6s	100%	100	100	100	100	100	100	100	100%
47	GPT-5.1	$0.013	13.8s	100%	100	100	100	100	100	100	100	100%
52	Claude Opus 4.6	$0.020	6.4s	100%	100	100	100	100	100	100	100	100%
94	MoonshotAI: Kimi K2.6	$0.012	1.2m	100%	100	100	100	100	100	100	100	100%
106	GPT-5	$0.034	43.1s	100%	100	100	100	100	100	100	100	100%
100	Gemma 4 26B (Reasoning)	$0.0017	1.8m	100%	100	100	100	100	100	100	100	100%
37	Grok 4.20 (Beta, Reasoning)	$0.012	6.4s	100%	100	100	100	100	100	100	100	100%
29	GPT-5.4 (Reasoning, Low)	$0.011	6.5s	100%	100	100	100	100	100	100	100	100%
28	Grok 4.20 (Reasoning)	$0.0060	18.4s	100%	100	100	100	100	100	100	100	100%
48	Z.AI GLM 5	$0.0044	34.8s	100%	100	100	100	100	100	100	100	100%
32	Claude Sonnet 4.6	$0.012	5.1s	100%	100	100	100	100	100	100	100	100%
103	MoonshotAI: Kimi K2.5	$0.0075	1.6m	100%	100	100	100	100	100	100	100	100%
45	ByteDance Seed 1.6	$0.0031	34.1s	100%	100	100	100	100	100	100	100	100%
24	Gemini 3 Flash (Preview, Reasoning)	$0.0065	10.3s	100%	100	100	100	100	100	100	100	100%
86	DeepSeek V4 Pro (Reasoning)	$0.0063	1.3m	100%	100	100	100	100	100	100	100	100%
51	Claude Opus 4.5	$0.020	5.7s	100%	100	100	100	100	100	100	100	100%
11	Grok 4.1 Fast	$0.0007	10.2s	100%	100	100	100	100	100	100	100	100%
61	Aion 2.0	$0.0040	49.3s	100%	100	100	100	100	100	100	100	100%
44	Z.AI GLM 4.6	$0.0051	27.9s	100%	100	100	100	100	100	100	100	100%
25	MiniMax M2.7	$0.0026	22.1s	100%	100	100	100	100	100	100	100	100%
50	GPT-5.5	$0.021	5.2s	100%	100	100	100	100	100	100	100	100%
15	DeepSeek V4 Flash (Reasoning)	$0.0004	16.6s	100%	100	100	100	100	100	100	100	100%
79	Gemini 3 Pro (Preview)	$0.027	17.9s	100%	100	100	100	100	100	100	100	100%
35	Claude Sonnet 4	$0.012	6.4s	100%	100	100	100	100	100	100	100	100%
63	Z.AI GLM 4.7	$0.0055	50.8s	100%	100	100	100	100	100	100	100	100%
18	GPT-4.1	$0.0059	4.6s	100%	100	100	100	100	100	100	100	100%
83	Gemini 2.5 Pro	$0.028	21.0s	100%	100	100	100	100	100	100	100	100%
72	Grok 4	$0.019	23.9s	100%	100	100	100	100	100	100	100	100%
33	Claude Sonnet 4.5	$0.012	5.2s	100%	100	100	100	100	100	100	100	100%
128	Claude Opus 4	$0.061	9.1s	100%	100	100	100	100	100	100	100	100%
12	Stealth: Hunter Alpha	$0.0000	12.6s	100%	100	100	100	100	100	100	100	100%
19	Gemma 4 31B	$0.0004	19.1s	100%	100	100	100	100	100	100	100	100%
16	Gemini 3.5 Flash (Reasoning, Minimal)	$0.0063	2.7s	100%	100	100	100	100	100	100	100	100%
3	Gemini 3.1 Flash Lite (Reasoning)	$0.0010	2.0s	100%	100	100	100	100	100	100	100	100%
43	Z.AI GLM 4.5	$0.0037	30.4s	100%	100	100	100	100	100	100	100	100%
5	Grok 4 Fast	$0.0007	5.0s	100%	100	100	100	100	100	100	100	100%
10	Qwen 3.5 Plus (2026-02-15)	$0.0017	7.6s	100%	100	100	100	100	100	100	100	100%
2	Gemini 3.1 Flash Lite (Preview)	$0.0010	1.9s	100%	100	100	100	100	100	100	100	100%
14	Gemma 4 26B	$0.0003	12.6s	100%	100	100	100	100	100	100	100	100%
6	Gemini 3.1 Flash Lite	$0.0010	5.9s	100%	100	100	100	100	100	100	100	100%
30	GPT-4o, May 13th (temp=0)	$0.012	4.6s	100%	100	100	100	100	100	100	100	100%
7	Gemini 3 Flash (Preview)	$0.0021	3.5s	100%	100	100	100	100	100	100	100	100%
13	Claude Haiku 4.5	$0.0041	3.1s	100%	100	100	100	100	100	100	100	100%
53	ByteDance Seed 2.0 Lite	$0.0040	46.7s	100%	100	100	100	100	100	100	100	100%
70	Claude 3.5 Sonnet	$0.024	10.3s	100%	100	100	100	100	100	100	100	100%
27	GPT-4o, May 13th (temp=1)	$0.012	3.9s	100%	100	100	100	100	100	100	100	100%
36	Claude 3.7 Sonnet	$0.012	6.6s	100%	100	100	100	100	100	100	100	100%
8	GPT-4.1 Mini	$0.0012	6.5s	100%	100	100	100	100	100	100	100	100%
26	Hermes 3 405B	$0.0013	25.6s	100%	100	100	100	100	100	100	100	100%
39	DeepSeek V4 Pro	$0.0020	32.0s	100%	100	100	100	100	100	100	100	100%
21	GPT-4o, Aug. 6th (temp=0)	$0.0074	2.7s	100%	100	100	100	100	100	100	100	100%
38	Qwen 3 32B	$0.0006	35.1s	100%	100	100	100	100	100	100	100	100%
4	Gemini 2.5 Flash	$0.0017	2.4s	100%	100	100	100	100	100	100	100	100%
20	Llama 3.1 Nemotron 70B	$0.0015	16.8s	100%	100	100	100	100	100	100	100	100%
1	Mistral Small Creative	$0.0002	3.2s	100%	100	100	100	100	100	100	100	100%
77	Z.AI GLM 5.1	$0.011	51.9s	100%	100	100	100	100	100	100	99	100%
74	Gemini 3.5 Flash (Reasoning)	$0.025	10.4s	100%	100	100	100	100	100	100	99	100%
69	GPT-5.5 (Reasoning)	$0.025	7.9s	100%	100	100	100	100	100	100	99	100%
68	o4 Mini High	$0.017	25.4s	100%	100	100	100	100	100	100	99	100%
9	Mistral Medium 3.1	$0.0014	6.9s	100%	100	100	100	100	100	100	99	100%
96	Qwen 3.5 122B	$0.021	49.1s	99%	100	100	100	100	100	99	99	100%
91	Qwen 3.6 27B	$0.013	1.0m	99%	100	100	100	100	100	99	99	100%
49	o4 Mini	$0.012	18.0s	99%	100	100	100	100	100	99	99	100%
59	Grok 4.3 (Reasoning)	$0.0074	38.1s	99%	100	100	100	100	99	99	99	100%
22	GPT-4o, Aug. 6th (temp=1)	$0.0074	3.5s	99%	100	100	100	100	99	99	99	100%
71	Gemma 4 31B (Reasoning)	$0.0007	1.1m	99%	100	100	100	99	99	99	99	100%
46	Qwen 3.6 35B	$0.0048	27.0s	99%	100	100	100	99	99	99	99	100%
76	Claude Opus 4.7 (Reasoning)	$0.028	4.6s	98%	100	100	100	100	100	100	97	100%
42	Qwen 3.6 Flash	$0.0062	18.8s	98%	100	100	99	99	99	99	99	99%
17	Stealth: Healer Alpha	$0.0000	13.9s	99%	100	99	99	99	99	99	99	99%
40	GPT-5.4	$0.010	6.0s	98%	100	100	100	100	99	99	97	99%
82	Qwen 3.5 27B	$0.012	53.2s	99%	99	99	99	99	99	99	99	99%
34	GPT-OSS 120B	$0.0007	29.7s	99%	99	99	99	99	99	99	99	99%
54	Z.AI GLM 4.5 Air	$0.0016	43.4s	97%	100	100	100	100	99	99	95	99%
31	Gemini 2.5 Flash (Reasoning)	$0.0055	8.3s	96%	99	99	99	99	99	99	95	99%
23	Inception Mercury 2	$0.0014	2.0s	95%	99	99	99	99	95	95	95	97%
98	MiniMax M2.5	$0.0017	1.1m	89%	100	99	99	99	99	97	85	97%
89	Claude Opus 4.7	$0.028	5.1s	94%	100	97	97	97	96	96	96	97%
84	Z.AI GLM 5 Turbo	$0.0086	18.8s	82%	100	100	100	100	100	100	74	96%
81	Llama 3.1 70B	$0.0005	34.1s	82%	100	100	100	100	100	100	74	96%
92	GPT-5.4 (Reasoning)	$0.014	11.6s	82%	100	100	100	100	100	99	74	96%
78	GPT-5.2	$0.010	7.0s	82%	100	100	100	100	99	99	74	96%
80	Xiaomi MIMO v2.5	$0.0052	20.7s	82%	100	100	100	100	99	99	74	96%
118	Qwen 3.5 Plus (2026-04-20)	$0.011	1.1m	81%	100	100	100	99	99	99	74	96%
138	Qwen 3.5 9B	$0.0014	2.1m	81%	100	100	100	99	99	99	74	96%
87	GPT-5 Mini	$0.0047	28.0s	81%	100	100	100	99	99	97	74	96%
67	ByteDance Seed 1.6 Flash	$0.0007	12.0s	81%	99	99	99	99	99	99	72	95%
57	Gemma 3 27B	$0.0002	17.7s	89%	98	98	93	93	93	93	93	94%
65	GPT-5.4 Nano (Reasoning, Low)	$0.0008	3.3s	80%	100	99	99	99	99	86	74	94%
73	Grok 4.20	$0.0021	4.5s	80%	100	100	100	100	100	83	74	94%
60	Inception Mercury	$0.0005	3.4s	84%	100	95	95	95	94	92	81	93%
145	Qwen 3.5 397B A17B	$0.010	2.5m	76%	100	100	100	100	100	74	74	92%
90	GPT-5.4 Mini (Reasoning)	$0.0061	5.8s	76%	100	100	100	100	100	74	74	92%
58	Qwen3 235B A22B Instruct 2507	$0.0003	9.7s	91%	91	91	91	91	91	91	91	91%
55	Arcee AI: Trinity Mini	$0.0002	7.4s	90%	91	91	91	91	91	90	90	91%
85	GPT-5.4 Nano	$0.0008	3.7s	75%	99	99	99	93	83	80	75	90%
141	Qwen 3.5 35B	$0.017	53.6s	70%	100	100	99	90	90	73	73	89%
64	Ministral 3 3B	$0.0001	2.2s	87%	91	91	91	91	88	85	85	89%
112	Nemotron 3 Super	$0.0000	47.0s	74%	99	99	95	94	90	73	73	89%
95	GPT-5.4 Mini (Reasoning, Low)	$0.0035	2.9s	74%	100	100	100	100	74	74	74	89%
97	Qwen 2.5 72B	$0.0003	11.6s	74%	100	100	100	100	74	74	74	89%
66	Gemma 3 4B	$0.0001	6.3s	89%	89	89	89	89	89	89	89	89%
125	GPT-5 Nano	$0.0028	58.3s	73%	100	100	99	97	75	75	73	89%
93	Writer: Palmyra X5	$0.0039	12.4s	80%	91	91	91	91	91	91	74	88%
75	Ministral 3B	$0.0001	2.2s	85%	88	88	88	88	85	85	85	87%
109	Grok 4.20 (Beta)	$0.0037	2.0s	62%	100	100	100	80	80	74	74	87%
124	Mistral Small 4 (Reasoning)	$0.0016	14.1s	55%	100	100	99	99	95	73	37	86%
108	Cydonia 24B V4.1	$0.0004	13.1s	65%	100	97	91	80	80	78	74	86%
126	Xiaomi MIMO v2.5 Pro	$0.0026	11.4s	55%	100	100	100	100	100	50	50	86%
104	GPT-5.4 Nano (Reasoning)	$0.0010	3.7s	68%	100	99	92	88	74	74	73	86%
129	DeepSeek-V2 Chat	$0.0009	16.4s	54%	100	100	100	74	74	74	74	85%
130	Mistral Large	$0.0048	8.3s	54%	100	100	100	74	74	74	74	85%
88	Claude 3 Haiku	$0.0010	4.8s	81%	86	86	86	86	86	86	78	85%
140	Qwen 3.5 Flash	$0.0037	57.2s	64%	100	99	91	83	73	73	73	85%
133	DeepSeek V3 (2025-03-24)	$0.0007	21.4s	56%	100	100	91	74	74	74	70	83%
144	Z.AI GLM 4.7 Flash	$0.0018	1.2m	55%	100	99	92	74	74	72	72	83%
135	DeepSeek V3 (2024-12-26)	$0.0008	16.6s	56%	100	100	74	74	74	74	74	81%
136	Hermes 3 70B	$0.0004	22.6s	56%	100	100	74	74	74	74	74	81%
102	GPT-5.4 Mini	$0.0031	2.3s	78%	83	83	83	83	83	80	74	81%
99	Mistral Small 3.2 24B	$0.0002	6.0s	80%	80	80	80	80	80	80	80	80%
107	Mistral Small 4	$0.0004	3.5s	72%	85	85	78	78	78	78	74	79%
111	Ministral 3 14B	$0.0003	4.4s	69%	88	88	78	78	76	74	74	79%
110	Skyfall 36B V2	$0.0007	10.7s	75%	83	80	80	80	78	74	74	78%
142	WizardLM 2 8x22b	$0.0008	42.8s	60%	100	74	74	74	74	74	74	77%
139	Gemini 2.5 Flash Lite (Reasoning)	$0.0026	19.1s	60%	100	74	74	74	74	73	73	77%
123	Llama 3.1 8B	$0.0001	7.1s	67%	86	74	74	74	74	74	73	75%
151	Nemotron 3 Nano	$0.0018	1.7m	42%	99	99	95	87	74	39	33	75%
115	Ministral 3 8B	$0.0002	3.5s	71%	78	78	74	74	74	74	74	75%
114	Ministral 8B	$0.0001	4.0s	73%	75	74	74	74	74	74	74	74%
119	Mistral Large 3	$0.0012	8.4s	74%	74	74	74	74	74	74	74	74%
127	Mistral Large 2	$0.0048	8.3s	74%	74	74	74	74	74	74	74	74%
137	DeepSeek V3.2	$0.0004	38.8s	74%	74	74	74	74	74	74	74	74%
116	DeepSeek V4 Flash	$0.0002	7.9s	74%	74	74	74	74	74	74	74	74%
122	GPT-4o Mini (temp=1)	$0.0004	11.9s	74%	74	74	74	74	74	74	74	74%
117	Gemma 3 12B	$0.0001	8.7s	74%	74	74	74	74	74	74	74	74%
120	GPT-4o Mini (temp=0)	$0.0004	10.7s	74%	74	74	74	74	74	74	74	74%
113	Mistral NeMO	$0.0002	1.8s	74%	74	74	74	74	74	74	74	74%
121	Grok 4.3	$0.0023	5.8s	73%	74	74	74	74	74	74	73	74%
146	ByteDance Seed 2.0 Mini	$0.0018	1.8m	73%	74	74	74	74	74	73	72	73%
143	Cohere Command R+ (Aug. 2024)	$0.0077	41.5s	69%	74	74	74	74	74	74	65	72%
132	LFM2 24B	$0.0001	16.0s	72%	72	72	72	72	72	72	72	72%
131	Gemini 2.5 Flash Lite	$0.0003	2.0s	67%	72	72	72	72	72	72	63	70%
148	DeepSeek V3.1	$0.0006	25.1s	38%	74	74	74	74	74	74	5	64%
149	Rocinante 12B	$0.0004	8.7s	29%	100	79	72	68	63	63	0	63%
150	Arcee AI: Trinity Large (Preview)	$0.0000	26.1s	36%	91	50	50	50	50	50	50	56%
147	GPT-4.1 Nano	$0.0003	4.5s	46%	63	51	51	51	51	51	50	52%
92.13%

Median	Evaluator	Top 3	Flop 3
100.0%	Dialogue content preserved	100Gemini 3 Flash (Preview) 100Gemma 4 31B 100Gemini 3 Flash (Preview, Reasoning)	83Rocinante 12B 84Nemotron 3 Nano 86DeepSeek V3.1
99.2%	Gender swap transformation accuracy	100Z.AI GLM 5 100Gemini 3.1 Pro (Preview) 100GPT-5.4 (Reasoning, Low)	9GPT-4.1 Nano 14Arcee AI: Trinity Large (Preview) 67Rocinante 12B
100.0%	Mara pronouns preserved (coreference test)	100Gemini 3 Flash (Preview, Reasoning) 100Gemini 3.1 Flash Lite (Preview) 100GPT-5.5	0Grok 4.3 0ByteDance Seed 2.0 Mini 0GPT-4.1 Nano
100.0%	Setting and description preserved	100Claude 3.5 Sonnet 100LFM2 24B 100GPT-OSS 120B	83Claude 3 Haiku 83Nemotron 3 Nano 86Rocinante 12B

Text Replacement

Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)