Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Text Replacement

Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.

Text Editing

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.1 Pro (Preview)	100%
Claude Sonnet 4.6 (Reasoning)	100%
GPT-5.1	100%
Claude Opus 4.6	100%
GPT-5	100%
Grok 4.20 (Beta, Reasoning)	100%
GPT-5.4 (Reasoning, Low)	100%
Z.AI GLM 5	100%
Claude Sonnet 4.6	100%
MoonshotAI: Kimi K2.5	100%
ByteDance Seed 1.6	100%
Gemini 3 Flash (Preview, Reasoning)	100%
Claude Opus 4.5	100%
Grok 4.1 Fast	100%
Aion 2.0	100%
Z.AI GLM 4.6	100%
MiniMax M2.7	100%
Gemini 3 Pro (Preview)	100%
Claude Sonnet 4	100%

	Score	Cost	Time
Mistral Small Creative	100%	$0.0002	3.2s
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s
Grok 4 Fast	100%	$0.0007	5.0s
GPT-5.4 Nano (Reasoning, Low)	94%	$0.0008	3.3s
Stealth: Hunter Alpha	100%	$0.0000	12.6s
Gemini 2.5 Flash	100%	$0.0017	2.4s
Grok 4.1 Fast	100%	$0.0007	10.2s
Mistral Medium 3.1	100%	$0.0014	6.9s
GPT-4.1 Mini	100%	$0.0012	6.5s
Inception Mercury 2	97%	$0.0014	2.0s
Qwen 2.5 72B	89%	$0.0003	11.6s
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s
Stealth: Healer Alpha	99%	$0.0000	13.9s
ByteDance Seed 1.6 Flash	95%	$0.0007	12.0s
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s
GPT-5.4 Mini (Reasoning, Low)	89%	$0.0035	2.9s
Inception Mercury	93%	$0.0005	3.4s
Llama 3.1 Nemotron 70B	100%	$0.0015	16.8s
Claude Haiku 4.5	100%	$0.0041	3.1s
Mistral Small 4 (Reasoning)	86%	$0.0016	14.1s

	Score	Cost	Speed	Stability
Mistral Small Creative	100%	$0.0002	3.2s	100%
Gemini 3.1 Flash Lite (Preview)	100%	$0.0010	1.9s	100%
Gemini 2.5 Flash	100%	$0.0017	2.4s	100%
Grok 4 Fast	100%	$0.0007	5.0s	100%
Gemini 3 Flash (Preview)	100%	$0.0021	3.5s	100%
GPT-4.1 Mini	100%	$0.0012	6.5s	100%
Mistral Medium 3.1	100%	$0.0014	6.9s	100%
Qwen 3.5 Plus (2026-02-15)	100%	$0.0017	7.6s	100%
Grok 4.1 Fast	100%	$0.0007	10.2s	100%
Stealth: Hunter Alpha	100%	$0.0000	12.6s	100%
Claude Haiku 4.5	100%	$0.0041	3.1s	100%
Stealth: Healer Alpha	99%	$0.0000	13.9s	99%
GPT-4.1	100%	$0.0059	4.6s	100%
Llama 3.1 Nemotron 70B	100%	$0.0015	16.8s	100%
GPT-4o, Aug. 6th (temp=0)	100%	$0.0074	2.7s	100%
GPT-4o, Aug. 6th (temp=1)	100%	$0.0074	3.5s	99%
Inception Mercury 2	97%	$0.0014	2.0s	95%
Gemini 3 Flash (Preview, Reasoning)	100%	$0.0065	10.3s	100%
MiniMax M2.7	100%	$0.0026	22.1s	100%
Hermes 3 405B	100%	$0.0013	25.6s	100%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	Total
47	Claude Opus 4.6 (Reasoning)	$0.021	7.3s	100%	100	100	100	100	100	100	100	100%
74	Gemini 3.1 Pro (Preview)	$0.032	31.1s	100%	100	100	100	100	100	100	100	100%
32	Claude Sonnet 4.6 (Reasoning)	$0.013	6.1s	100%	100	100	100	100	100	100	100	100%
36	GPT-5.1	$0.013	13.8s	100%	100	100	100	100	100	100	100	100%
40	Claude Opus 4.6	$0.020	6.4s	100%	100	100	100	100	100	100	100	100%
78	GPT-5	$0.034	43.1s	100%	100	100	100	100	100	100	100	100%
29	Grok 4.20 (Beta, Reasoning)	$0.012	6.4s	100%	100	100	100	100	100	100	100	100%
22	GPT-5.4 (Reasoning, Low)	$0.011	6.5s	100%	100	100	100	100	100	100	100	100%
37	Z.AI GLM 5	$0.0044	34.8s	100%	100	100	100	100	100	100	100	100%
25	Claude Sonnet 4.6	$0.012	5.1s	100%	100	100	100	100	100	100	100	100%
76	MoonshotAI: Kimi K2.5	$0.0075	1.6m	100%	100	100	100	100	100	100	100	100%
35	ByteDance Seed 1.6	$0.0031	34.1s	100%	100	100	100	100	100	100	100	100%
18	Gemini 3 Flash (Preview, Reasoning)	$0.0065	10.3s	100%	100	100	100	100	100	100	100	100%
39	Claude Opus 4.5	$0.020	5.7s	100%	100	100	100	100	100	100	100	100%
9	Grok 4.1 Fast	$0.0007	10.2s	100%	100	100	100	100	100	100	100	100%
46	Aion 2.0	$0.0040	49.3s	100%	100	100	100	100	100	100	100	100%
34	Z.AI GLM 4.6	$0.0051	27.9s	100%	100	100	100	100	100	100	100	100%
19	MiniMax M2.7	$0.0026	22.1s	100%	100	100	100	100	100	100	100	100%
58	Gemini 3 Pro (Preview)	$0.027	17.9s	100%	100	100	100	100	100	100	100	100%
27	Claude Sonnet 4	$0.012	6.4s	100%	100	100	100	100	100	100	100	100%
48	Z.AI GLM 4.7	$0.0055	50.8s	100%	100	100	100	100	100	100	100	100%
13	GPT-4.1	$0.0059	4.6s	100%	100	100	100	100	100	100	100	100%
61	Gemini 2.5 Pro	$0.028	21.0s	100%	100	100	100	100	100	100	100	100%
55	Grok 4	$0.019	23.9s	100%	100	100	100	100	100	100	100	100%
26	Claude Sonnet 4.5	$0.012	5.2s	100%	100	100	100	100	100	100	100	100%
94	Claude Opus 4	$0.061	9.1s	100%	100	100	100	100	100	100	100	100%
10	Stealth: Hunter Alpha	$0.0000	12.6s	100%	100	100	100	100	100	100	100	100%
33	Z.AI GLM 4.5	$0.0037	30.4s	100%	100	100	100	100	100	100	100	100%
4	Grok 4 Fast	$0.0007	5.0s	100%	100	100	100	100	100	100	100	100%
8	Qwen 3.5 Plus (2026-02-15)	$0.0017	7.6s	100%	100	100	100	100	100	100	100	100%
2	Gemini 3.1 Flash Lite (Preview)	$0.0010	1.9s	100%	100	100	100	100	100	100	100	100%
23	GPT-4o, May 13th (temp=0)	$0.012	4.6s	100%	100	100	100	100	100	100	100	100%
5	Gemini 3 Flash (Preview)	$0.0021	3.5s	100%	100	100	100	100	100	100	100	100%
11	Claude Haiku 4.5	$0.0041	3.1s	100%	100	100	100	100	100	100	100	100%
41	ByteDance Seed 2.0 Lite	$0.0040	46.7s	100%	100	100	100	100	100	100	100	100%
54	Claude 3.5 Sonnet	$0.024	10.3s	100%	100	100	100	100	100	100	100	100%
21	GPT-4o, May 13th (temp=1)	$0.012	3.9s	100%	100	100	100	100	100	100	100	100%
28	Claude 3.7 Sonnet	$0.012	6.6s	100%	100	100	100	100	100	100	100	100%
6	GPT-4.1 Mini	$0.0012	6.5s	100%	100	100	100	100	100	100	100	100%
20	Hermes 3 405B	$0.0013	25.6s	100%	100	100	100	100	100	100	100	100%
15	GPT-4o, Aug. 6th (temp=0)	$0.0074	2.7s	100%	100	100	100	100	100	100	100	100%
30	Qwen 3 32B	$0.0006	35.1s	100%	100	100	100	100	100	100	100	100%
3	Gemini 2.5 Flash	$0.0017	2.4s	100%	100	100	100	100	100	100	100	100%
14	Llama 3.1 Nemotron 70B	$0.0015	16.8s	100%	100	100	100	100	100	100	100	100%
1	Mistral Small Creative	$0.0002	3.2s	100%	100	100	100	100	100	100	100	100%
53	o4 Mini High	$0.017	25.4s	100%	100	100	100	100	100	100	99	100%
7	Mistral Medium 3.1	$0.0014	6.9s	100%	100	100	100	100	100	100	99	100%
70	Qwen 3.5 122B	$0.021	49.1s	99%	100	100	100	100	100	99	99	100%
38	o4 Mini	$0.012	18.0s	99%	100	100	100	100	100	99	99	100%
16	GPT-4o, Aug. 6th (temp=1)	$0.0074	3.5s	99%	100	100	100	100	99	99	99	100%
12	Stealth: Healer Alpha	$0.0000	13.9s	99%	100	99	99	99	99	99	99	99%
31	GPT-5.4	$0.010	6.0s	98%	100	100	100	100	99	99	97	99%
60	Qwen 3.5 27B	$0.012	53.2s	99%	99	99	99	99	99	99	99	99%
24	Gemini 2.5 Flash (Reasoning)	$0.0055	8.3s	96%	99	99	99	99	99	99	95	99%
17	Inception Mercury 2	$0.0014	2.0s	95%	99	99	99	99	95	95	95	97%
72	MiniMax M2.5	$0.0017	1.1m	89%	100	99	99	99	99	97	85	97%
62	Z.AI GLM 5 Turbo	$0.0086	18.8s	82%	100	100	100	100	100	100	74	96%
59	Llama 3.1 70B	$0.0005	34.1s	82%	100	100	100	100	100	100	74	96%
67	GPT-5.4 (Reasoning)	$0.014	11.6s	82%	100	100	100	100	100	99	74	96%
57	GPT-5.2	$0.010	7.0s	82%	100	100	100	100	99	99	74	96%
103	Qwen 3.5 9B	$0.0014	2.1m	81%	100	100	100	99	99	99	74	96%
64	GPT-5 Mini	$0.0047	28.0s	81%	100	100	100	99	99	97	74	96%
52	ByteDance Seed 1.6 Flash	$0.0007	12.0s	81%	99	99	99	99	99	99	72	95%
43	Gemma 3 27B	$0.0002	17.7s	89%	98	98	93	93	93	93	93	94%
50	GPT-5.4 Nano (Reasoning, Low)	$0.0008	3.3s	80%	100	99	99	99	99	86	74	94%
45	Inception Mercury	$0.0005	3.4s	84%	100	95	95	95	94	92	81	93%
110	Qwen 3.5 397B A17B	$0.010	2.5m	76%	100	100	100	100	100	74	74	92%
66	GPT-5.4 Mini (Reasoning)	$0.0061	5.8s	76%	100	100	100	100	100	74	74	92%
44	Qwen3 235B A22B Instruct 2507	$0.0003	9.7s	91%	91	91	91	91	91	91	91	91%
42	Arcee AI: Trinity Mini	$0.0002	7.4s	90%	91	91	91	91	91	90	90	91%
63	GPT-5.4 Nano	$0.0008	3.7s	75%	99	99	99	93	83	80	75	90%
106	Qwen 3.5 35B	$0.017	53.6s	70%	100	100	99	90	90	73	73	89%
49	Ministral 3 3B	$0.0001	2.2s	87%	91	91	91	91	88	85	85	89%
82	Nemotron 3 Super	$0.0000	47.0s	74%	99	99	95	94	90	73	73	89%
69	GPT-5.4 Mini (Reasoning, Low)	$0.0035	2.9s	74%	100	100	100	100	74	74	74	89%
71	Qwen 2.5 72B	$0.0003	11.6s	74%	100	100	100	100	74	74	74	89%
51	Gemma 3 4B	$0.0001	6.3s	89%	89	89	89	89	89	89	89	89%
92	GPT-5 Nano	$0.0028	58.3s	73%	100	100	99	97	75	75	73	89%
68	Writer: Palmyra X5	$0.0039	12.4s	80%	91	91	91	91	91	91	74	88%
56	Ministral 3B	$0.0001	2.2s	85%	88	88	88	88	85	85	85	87%
80	Grok 4.20 (Beta)	$0.0037	2.0s	62%	100	100	100	80	80	74	74	87%
91	Mistral Small 4 (Reasoning)	$0.0016	14.1s	55%	100	100	99	99	95	73	37	86%
77	GPT-5.4 Nano (Reasoning)	$0.0010	3.7s	68%	100	99	92	88	74	74	73	86%
95	DeepSeek-V2 Chat	$0.0009	16.4s	54%	100	100	100	74	74	74	74	85%
96	Mistral Large	$0.0048	8.3s	54%	100	100	100	74	74	74	74	85%
65	Claude 3 Haiku	$0.0010	4.8s	81%	86	86	86	86	86	86	78	85%
105	Qwen 3.5 Flash	$0.0037	57.2s	64%	100	99	91	83	73	73	73	85%
99	DeepSeek V3 (2025-03-24)	$0.0007	21.4s	56%	100	100	91	74	74	74	70	83%
109	Z.AI GLM 4.7 Flash	$0.0018	1.2m	55%	100	99	92	74	74	72	72	83%
100	DeepSeek V3 (2024-12-26)	$0.0008	16.6s	56%	100	100	74	74	74	74	74	81%
101	Hermes 3 70B	$0.0004	22.6s	56%	100	100	74	74	74	74	74	81%
75	GPT-5.4 Mini	$0.0031	2.3s	78%	83	83	83	83	83	80	74	81%
73	Mistral Small 3.2 24B	$0.0002	6.0s	80%	80	80	80	80	80	80	80	80%
79	Mistral Small 4	$0.0004	3.5s	72%	85	85	78	78	78	78	74	79%
81	Ministral 3 14B	$0.0003	4.4s	69%	88	88	78	78	76	74	74	79%
107	WizardLM 2 8x22b	$0.0008	42.8s	60%	100	74	74	74	74	74	74	77%
104	Gemini 2.5 Flash Lite (Reasoning)	$0.0026	19.1s	60%	100	74	74	74	74	73	73	77%
90	Llama 3.1 8B	$0.0001	7.1s	67%	86	74	74	74	74	74	73	75%
116	Nemotron 3 Nano	$0.0018	1.7m	42%	99	99	95	87	74	39	33	75%
85	Ministral 3 8B	$0.0002	3.5s	71%	78	78	74	74	74	74	74	75%
84	Ministral 8B	$0.0001	4.0s	73%	75	74	74	74	74	74	74	74%
87	Mistral Large 3	$0.0012	8.4s	74%	74	74	74	74	74	74	74	74%
93	Mistral Large 2	$0.0048	8.3s	74%	74	74	74	74	74	74	74	74%
102	DeepSeek V3.2	$0.0004	38.8s	74%	74	74	74	74	74	74	74	74%
89	GPT-4o Mini (temp=1)	$0.0004	11.9s	74%	74	74	74	74	74	74	74	74%
86	Gemma 3 12B	$0.0001	8.7s	74%	74	74	74	74	74	74	74	74%
88	GPT-4o Mini (temp=0)	$0.0004	10.7s	74%	74	74	74	74	74	74	74	74%
83	Mistral NeMO	$0.0002	1.8s	74%	74	74	74	74	74	74	74	74%
111	ByteDance Seed 2.0 Mini	$0.0018	1.8m	73%	74	74	74	74	74	73	72	73%
108	Cohere Command R+ (Aug. 2024)	$0.0077	41.5s	69%	74	74	74	74	74	74	65	72%
98	LFM2 24B	$0.0001	16.0s	72%	72	72	72	72	72	72	72	72%
97	Gemini 2.5 Flash Lite	$0.0003	2.0s	67%	72	72	72	72	72	72	63	70%
113	DeepSeek V3.1	$0.0006	25.1s	38%	74	74	74	74	74	74	5	64%
114	Rocinante 12B	$0.0004	8.7s	29%	100	79	72	68	63	63	0	63%
115	Arcee AI: Trinity Large (Preview)	$0.0000	26.1s	36%	91	50	50	50	50	50	50	56%
112	GPT-4.1 Nano	$0.0003	4.5s	46%	63	51	51	51	51	51	50	52%
90.83%

Median	Evaluator	Top 3	Flop 3
100.0%	Dialogue content preserved	100Qwen 2.5 72B 100MiniMax M2.7 100GPT-5.4 (Reasoning, Low)	83Rocinante 12B 84Nemotron 3 Nano 86DeepSeek V3.1
99.0%	Gender swap transformation accuracy	100Llama 3.1 Nemotron 70B 100Gemini 3 Flash (Preview) 100GPT-4o, May 13th (temp=0)	9GPT-4.1 Nano 14Arcee AI: Trinity Large (Preview) 67Rocinante 12B
85.7%	Mara pronouns preserved (coreference test)	100Gemini 3 Pro (Preview) 100Gemini 3 Flash (Preview, Reasoning) 100GPT-4.1 Mini	0LFM2 24B 0DeepSeek V3.2 0GPT-4o Mini (temp=0)
100.0%	Setting and description preserved	100GPT-5 100ByteDance Seed 1.6 100Z.AI GLM 4.5	83Nemotron 3 Nano 83Claude 3 Haiku 86Rocinante 12B

Text Replacement

Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)