Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Text Replacement

Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.

Text Editing

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Claude Opus 4.6 (Reasoning)	100%
Gemini 3.1 Pro (Preview)	100%
Claude Sonnet 4.6 (Reasoning)	100%
GPT-5.1	100%
Claude Opus 4.6	100%
GPT-5	100%
Qwen 3.5 397B A17B	100%
Z.AI GLM 5	100%
Claude Sonnet 4.6	100%
ByteDance Seed 1.6	100%
Claude Opus 4.5	100%
Aion 2.0	100%
Gemini 3 Pro (Preview)	100%
Claude Sonnet 4	100%
Z.AI GLM 4.7	100%
GPT-4.1	100%
Gemini 2.5 Pro	100%
Grok 4	100%
Claude Sonnet 4.5	100%
Claude Opus 4	100%

	Score	Cost	Time
Ministral 3B	99%	$0.0001	2.5s
Mistral NeMO	100%	$0.0002	3.6s
Ministral 3 3B	100%	$0.0001	2.2s
Gemini 2.5 Flash Lite	100%	$0.0003	1.9s
Ministral 3 8B	100%	$0.0002	3.9s
Mistral Small Creative	100%	$0.0003	3.2s
Ministral 8B	100%	$0.0001	3.8s
GPT-4.1 Nano	95%	$0.0003	3.9s
Ministral 3 14B	100%	$0.0003	4.6s
Mistral Small 3.2 24B	86%	$0.0003	4.9s
Gemma 3 4B	100%	$0.0001	6.6s
Gemini 2.5 Flash	100%	$0.0017	2.4s
Claude 3 Haiku	100%	$0.0011	5.2s
Llama 3.1 8B	100%	$0.0000	13.0s
Grok 4 Fast	100%	$0.0010	7.0s
Qwen 2.5 72B	100%	$0.0003	11.4s
Gemini 3 Flash (Preview)	100%	$0.0022	3.6s
GPT-4.1 Mini	100%	$0.0012	7.8s
Mistral Medium 3.1	99%	$0.0015	6.1s
Mistral Large 3	99%	$0.0013	8.3s

	Score	Cost	Speed	Stability
Ministral 3 3B	100%	$0.0001	2.2s	100%
Gemini 2.5 Flash Lite	100%	$0.0003	1.9s	100%
Mistral Small Creative	100%	$0.0003	3.2s	100%
Mistral NeMO	100%	$0.0002	3.6s	100%
Ministral 8B	100%	$0.0001	3.8s	100%
Ministral 3 8B	100%	$0.0002	3.9s	100%
Ministral 3 14B	100%	$0.0003	4.6s	100%
Gemini 2.5 Flash	100%	$0.0017	2.4s	100%
Gemma 3 4B	100%	$0.0001	6.6s	100%
Claude 3 Haiku	100%	$0.0011	5.2s	100%
Gemini 3 Flash (Preview)	100%	$0.0022	3.6s	100%
Grok 4 Fast	100%	$0.0010	7.0s	100%
Ministral 3B	99%	$0.0001	2.5s	95%
GPT-4.1 Mini	100%	$0.0012	7.8s	99%
Claude Haiku 4.5	100%	$0.0043	3.4s	100%
Qwen 2.5 72B	100%	$0.0003	11.4s	100%
Mistral Large 3	99%	$0.0013	8.3s	99%
Mistral Medium 3.1	99%	$0.0015	6.1s	97%
GPT-4.1	100%	$0.0062	4.9s	100%
Llama 3.1 8B	100%	$0.0000	13.0s	98%

Median	Evaluator	Top 3	Flop 3
100.0%	Dialogue content preserved	100Claude 3.5 Sonnet 100Ministral 3B 100Mistral Large 2	56Hermes 3 70B 81Rocinante 12B 86Gemini 2.5 Flash Lite (Reasoning)
100.0%	Gender swap transformation accuracy	100Gemini 2.5 Flash (Reasoning) 100Hermes 3 405B 100Claude Haiku 4.5	46Hermes 3 70B 84Rocinante 12B 85Gemini 2.5 Flash Lite (Reasoning)
100.0%	Mara pronouns preserved (coreference test)	100Z.AI GLM 5 100o4 Mini High 100Claude Haiku 4.5	43Hermes 3 70B 58Gemma 3 12B 64GPT-4o Mini (temp=1)
100.0%	Setting and description preserved	100Z.AI GLM 4.6 100GPT-4.1 Mini 100Claude Sonnet 4	57Hermes 3 70B 86Rocinante 12B 86Mistral Small 3.2 24B

Text Replacement

Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	Total
82	Claude Opus 4.6 (Reasoning)	$0.064	26.3s	100%	100	100	100	100	100	100	100	100%
73	Gemini 3.1 Pro (Preview)	$0.044	41.3s	100%	100	100	100	100	100	100	100	100%
83	Claude Sonnet 4.6 (Reasoning)	$0.063	34.1s	100%	100	100	100	100	100	100	100	100%
60	GPT-5.1	$0.021	19.3s	100%	100	100	100	100	100	100	100	100%
55	Claude Opus 4.6	$0.021	6.5s	100%	100	100	100	100	100	100	100	100%
74	GPT-5	$0.037	56.3s	100%	100	100	100	100	100	100	100	100%
72	Qwen 3.5 397B A17B	$0.0050	1.8m	100%	100	100	100	100	100	100	100	100%
68	Z.AI GLM 5	$0.010	1.2m	100%	100	100	100	100	100	100	100	100%
38	Claude Sonnet 4.6	$0.013	5.1s	100%	100	100	100	100	100	100	100	100%
56	ByteDance Seed 1.6	$0.0039	41.3s	100%	100	100	100	100	100	100	100	100%
54	Claude Opus 4.5	$0.021	5.8s	100%	100	100	100	100	100	100	100	100%
57	Aion 2.0	$0.0036	45.8s	100%	100	100	100	100	100	100	100	100%
67	Gemini 3 Pro (Preview)	$0.036	24.8s	100%	100	100	100	100	100	100	100	100%
41	Claude Sonnet 4	$0.013	6.5s	100%	100	100	100	100	100	100	100	100%
65	Z.AI GLM 4.7	$0.0062	1.3m	100%	100	100	100	100	100	100	100	100%
19	GPT-4.1	$0.0062	4.9s	100%	100	100	100	100	100	100	100	100%
63	Gemini 2.5 Pro	$0.027	19.9s	100%	100	100	100	100	100	100	100	100%
70	Grok 4	$0.031	37.9s	100%	100	100	100	100	100	100	100	100%
39	Claude Sonnet 4.5	$0.013	5.3s	100%	100	100	100	100	100	100	100	100%
76	Claude Opus 4	$0.064	8.8s	100%	100	100	100	100	100	100	100	100%
29	Gemini 2.5 Flash (Reasoning)	$0.0064	9.2s	100%	100	100	100	100	100	100	100	100%
51	Z.AI GLM 4.5	$0.0044	34.4s	100%	100	100	100	100	100	100	100	100%
12	Grok 4 Fast	$0.0010	7.0s	100%	100	100	100	100	100	100	100	100%
35	GPT-4o, May 13th (temp=0)	$0.013	3.5s	100%	100	100	100	100	100	100	100	100%
11	Gemini 3 Flash (Preview)	$0.0022	3.6s	100%	100	100	100	100	100	100	100	100%
15	Claude Haiku 4.5	$0.0043	3.4s	100%	100	100	100	100	100	100	100	100%
27	DeepSeek-V2 Chat	$0.0009	19.1s	100%	100	100	100	100	100	100	100	100%
61	Claude 3.5 Sonnet	$0.026	11.2s	100%	100	100	100	100	100	100	100	100%
36	GPT-4o, May 13th (temp=1)	$0.013	4.1s	100%	100	100	100	100	100	100	100	100%
31	DeepSeek V3 (2024-12-26)	$0.0009	20.0s	100%	100	100	100	100	100	100	100	100%
40	Claude 3.7 Sonnet	$0.013	6.3s	100%	100	100	100	100	100	100	100	100%
34	Hermes 3 405B	$0.0014	24.6s	100%	100	100	100	100	100	100	100	100%
22	GPT-4o, Aug. 6th (temp=1)	$0.0078	3.0s	100%	100	100	100	100	100	100	100	100%
21	GPT-4o, Aug. 6th (temp=0)	$0.0078	3.0s	100%	100	100	100	100	100	100	100	100%
50	DeepSeek V3.2	$0.0008	39.0s	100%	100	100	100	100	100	100	100	100%
2	Gemini 2.5 Flash Lite	$0.0003	1.9s	100%	100	100	100	100	100	100	100	100%
8	Gemini 2.5 Flash	$0.0017	2.4s	100%	100	100	100	100	100	100	100	100%
30	Writer: Palmyra X5	$0.0041	13.7s	100%	100	100	100	100	100	100	100	100%
47	Llama 3.1 70B	$0.0006	38.2s	100%	100	100	100	100	100	100	100	100%
23	Gemma 3 27B	$0.0003	18.4s	100%	100	100	100	100	100	100	100	100%
16	Qwen 2.5 72B	$0.0003	11.4s	100%	100	100	100	100	100	100	100	100%
37	Arcee AI: Trinity Large (Preview)	$0.0000	28.0s	100%	100	100	100	100	100	100	100	100%
3	Mistral Small Creative	$0.0003	3.2s	100%	100	100	100	100	100	100	100	100%
7	Ministral 3 14B	$0.0003	4.6s	100%	100	100	100	100	100	100	100	100%
6	Ministral 3 8B	$0.0002	3.9s	100%	100	100	100	100	100	100	100	100%
10	Claude 3 Haiku	$0.0011	5.2s	100%	100	100	100	100	100	100	100	100%
44	WizardLM 2 8x22b	$0.0009	37.3s	100%	100	100	100	100	100	100	100	100%
9	Gemma 3 4B	$0.0001	6.6s	100%	100	100	100	100	100	100	100	100%
1	Ministral 3 3B	$0.0001	2.2s	100%	100	100	100	100	100	100	100	100%
4	Mistral NeMO	$0.0002	3.6s	100%	100	100	100	100	100	100	100	100%
5	Ministral 8B	$0.0001	3.8s	100%	100	100	100	100	100	100	100	100%
80	MoonshotAI: Kimi K2.5	$0.0095	2.0m	100%	100	100	100	100	100	100	99	100%
59	o4 Mini	$0.017	25.0s	100%	100	100	100	100	100	100	99	100%
26	Llama 3.1 Nemotron 70B	$0.0017	15.8s	100%	100	100	100	100	100	100	99	100%
69	o4 Mini High	$0.028	40.6s	99%	100	100	100	100	100	99	99	100%
14	GPT-4.1 Mini	$0.0012	7.8s	99%	100	100	100	100	100	99	99	100%
48	Cohere Command R+ (Aug. 2024)	$0.0081	23.6s	99%	100	100	100	100	100	99	99	100%
25	Mistral Large 2	$0.0051	8.5s	99%	100	100	100	100	99	99	99	100%
58	GPT-5 Mini	$0.0067	37.7s	98%	100	100	100	100	100	100	98	100%
42	Gemini 3 Flash (Preview, Reasoning)	$0.010	16.3s	98%	100	100	100	100	100	100	98	100%
20	Llama 3.1 8B	$0.0000	13.0s	98%	100	100	100	100	100	99	97	100%
28	Mistral Large	$0.0051	8.5s	99%	100	100	99	99	99	99	99	100%
24	Grok 4.1 Fast	$0.0011	11.6s	97%	100	100	100	100	100	100	95	99%
71	Z.AI GLM 4.6	$0.0048	1.5m	97%	100	100	100	100	100	100	95	99%
17	Mistral Large 3	$0.0013	8.3s	99%	99	99	99	99	99	99	99	99%
46	DeepSeek V3 (2025-03-24)	$0.0006	31.8s	97%	100	100	100	100	100	100	95	99%
77	Minimax M2.5	$0.0016	2.0m	99%	99	99	99	99	99	99	99	99%
13	Ministral 3B	$0.0001	2.5s	95%	100	100	100	100	100	100	93	99%
18	Mistral Medium 3.1	$0.0015	6.1s	97%	100	100	100	100	97	97	97	99%
33	ByteDance Seed 1.6 Flash	$0.0007	12.5s	94%	100	100	100	100	100	100	91	99%
32	Qwen 3.5 Plus (2026-02-15)	$0.0018	7.9s	93%	100	100	100	100	100	100	90	99%
43	GPT-5.2	$0.0099	7.1s	94%	100	100	100	97	97	97	95	98%
66	GPT-5 Nano	$0.0031	1.1m	91%	100	99	99	95	95	95	95	97%
64	Z.AI GLM 4.7 Flash	$0.0015	59.8s	91%	98	97	97	95	95	95	92	95%
45	GPT-4.1 Nano	$0.0003	3.9s	82%	100	100	100	100	95	95	74	95%
78	DeepSeek V3.1	$0.0006	1.0m	62%	100	100	100	100	100	96	45	92%
52	GPT-4o Mini (temp=1)	$0.0005	10.2s	91%	91	91	91	91	91	91	91	91%
53	GPT-4o Mini (temp=0)	$0.0005	10.8s	91%	91	91	91	91	91	91	91	91%
49	Arcee AI: Trinity Mini	$0.0002	6.8s	91%	91	91	91	91	91	91	91	91%
62	Gemma 3 12B	$0.0001	10.5s	81%	100	89	89	89	89	86	86	90%
75	Mistral Small 3.2 24B	$0.0003	4.9s	30%	100	100	100	100	100	100	0	86%
79	Rocinante 12B	$0.0004	9.4s	31%	100	100	100	98	97	87	0	83%
81	Gemini 2.5 Flash Lite (Reasoning)	$0.0019	11.9s	32%	100	100	99	99	93	86	0	83%
84	Hermes 3 70B	$0.0005	19.7s	4%	100	100	100	52	0	0	0	50%
97.97%