Text Replacement
Tests deterministic text transformations: renaming characters/locations, expanding contractions, tense rewriting, POV shifts, gender swaps, combined transformations, and word avoidance. Scored by checking each expected change independently.
Multi-character gender swap: Priya(F)->Rohan(M), Mara unchanged
Text Editing
Performance Score Distribution (Top 20)
Click a model name to view its detail page.
| Score | ||
|---|---|---|
| Claude Opus 4.6 (Reasoning) | 100% | |
| Gemini 3.1 Pro (Preview) | 100% | |
| Claude Sonnet 4.6 (Reasoning) | 100% | |
| GPT-5.1 | 100% | |
| Claude Opus 4.6 | 100% | |
| GPT-5 | 100% | |
| Z.AI GLM 5 | 100% | |
| Claude Sonnet 4.6 | 100% | |
| MoonshotAI: Kimi K2.5 | 100% | |
| ByteDance Seed 1.6 | 100% | |
| Gemini 3 Flash (Preview, Reasoning) | 100% | |
| Claude Opus 4.5 | 100% | |
| Grok 4.1 Fast | 100% | |
| Aion 2.0 | 100% | |
| Z.AI GLM 4.6 | 100% | |
| Gemini 3 Pro (Preview) | 100% | |
| Claude Sonnet 4 | 100% | |
| Z.AI GLM 4.7 | 100% | |
| GPT-4.1 | 100% | |
| Gemini 2.5 Pro | 100% | |
Price-Performance Score Distribution (Top 20)
Click a model name to view its detail page.
| Score | Cost | Time | ||
|---|---|---|---|---|
| Mistral Small Creative | 100% | $0.0002 | 3.2s | |
| Grok 4 Fast | 100% | $0.0007 | 5.0s | |
| Gemini 2.5 Flash | 100% | $0.0017 | 2.4s | |
| Mistral Medium 3.1 | 100% | $0.0014 | 6.9s | |
| GPT-4.1 Mini | 100% | $0.0012 | 6.5s | |
| Grok 4.1 Fast | 100% | $0.0007 | 10.2s | |
| Gemini 3 Flash (Preview) | 100% | $0.0021 | 3.5s | |
| Qwen 2.5 72B | 89% | $0.0003 | 11.6s | |
| ByteDance Seed 1.6 Flash | 95% | $0.0007 | 12.0s | |
| Qwen 3.5 Plus (2026-02-15) | 100% | $0.0017 | 7.6s | |
| Claude Haiku 4.5 | 100% | $0.0041 | 3.1s | |
| Llama 3.1 Nemotron 70B | 100% | $0.0015 | 16.8s | |
| Ministral 3 3B | 89% | $0.0001 | 2.2s | |
| Llama 3.1 70B | 96% | $0.0005 | 34.1s | |
| GPT-4.1 | 100% | $0.0059 | 4.6s | |
| Hermes 3 405B | 100% | $0.0013 | 25.6s | |
| Gemma 3 27B | 94% | $0.0002 | 17.7s | |
| Arcee AI: Trinity Mini | 91% | $0.0002 | 7.4s | |
| GPT-4o, Aug. 6th (temp=0) | 100% | $0.0074 | 2.7s | |
| Ministral 3B | 87% | $0.0001 | 2.2s | |
Most Stable Models (Top 20)
Ranked by stability (median × consistency). Click a model name to view its detail page.
| Score | Consistency | Stability | ||
|---|---|---|---|---|
| Claude Opus 4.6 (Reasoning) | 100% | 100% | 100% | |
| Gemini 3.1 Pro (Preview) | 100% | 100% | 100% | |
| Claude Sonnet 4.6 (Reasoning) | 100% | 100% | 100% | |
| GPT-5.1 | 100% | 100% | 100% | |
| Claude Opus 4.6 | 100% | 100% | 100% | |
| GPT-5 | 100% | 100% | 100% | |
| Z.AI GLM 5 | 100% | 100% | 100% | |
| Claude Sonnet 4.6 | 100% | 100% | 100% | |
| MoonshotAI: Kimi K2.5 | 100% | 100% | 100% | |
| ByteDance Seed 1.6 | 100% | 100% | 100% | |
| Gemini 3 Flash (Preview, Reasoning) | 100% | 100% | 100% | |
| Claude Opus 4.5 | 100% | 100% | 100% | |
| Grok 4.1 Fast | 100% | 100% | 100% | |
| Aion 2.0 | 100% | 100% | 100% | |
| Z.AI GLM 4.6 | 100% | 100% | 100% | |
| Gemini 3 Pro (Preview) | 100% | 100% | 100% | |
| Claude Sonnet 4 | 100% | 100% | 100% | |
| Z.AI GLM 4.7 | 100% | 100% | 100% | |
| GPT-4.1 | 100% | 100% | 100% | |
| Gemini 2.5 Pro | 100% | 100% | 100% | |
Top Overall Models (Top 20)
Ranked by composite score (performance, cost, speed & stability). Click a model name to view its detail page.
| Score | Cost | Speed | Stability | ||
|---|---|---|---|---|---|
| Mistral Small Creative | 100% | $0.0002 | 3.2s | 100% | |
| Gemini 2.5 Flash | 100% | $0.0017 | 2.4s | 100% | |
| Grok 4 Fast | 100% | $0.0007 | 5.0s | 100% | |
| Gemini 3 Flash (Preview) | 100% | $0.0021 | 3.5s | 100% | |
| GPT-4.1 Mini | 100% | $0.0012 | 6.5s | 100% | |
| Mistral Medium 3.1 | 100% | $0.0014 | 6.9s | 100% | |
| Qwen 3.5 Plus (2026-02-15) | 100% | $0.0017 | 7.6s | 100% | |
| Grok 4.1 Fast | 100% | $0.0007 | 10.2s | 100% | |
| Claude Haiku 4.5 | 100% | $0.0041 | 3.1s | 100% | |
| GPT-4.1 | 100% | $0.0059 | 4.6s | 100% | |
| Llama 3.1 Nemotron 70B | 100% | $0.0015 | 16.8s | 100% | |
| GPT-4o, Aug. 6th (temp=0) | 100% | $0.0074 | 2.7s | 100% | |
| GPT-4o, Aug. 6th (temp=1) | 100% | $0.0074 | 3.5s | 99% | |
| Gemini 3 Flash (Preview, Reasoning) | 100% | $0.0065 | 10.3s | 100% | |
| Hermes 3 405B | 100% | $0.0013 | 25.6s | 100% | |
| GPT-4o, May 13th (temp=1) | 100% | $0.012 | 3.9s | 100% | |
| GPT-4o, May 13th (temp=0) | 100% | $0.012 | 4.6s | 100% | |
| Gemini 2.5 Flash (Reasoning) | 99% | $0.0055 | 8.3s | 96% | |
| Claude Sonnet 4.6 | 100% | $0.012 | 5.1s | 100% | |
| Claude Sonnet 4.5 | 100% | $0.012 | 5.2s | 100% | |
| Median | Evaluator | Top 3 | Flop 3 |
|---|---|---|---|
| 100.0% | Dialogue content preserved | ||
| 99.4% | Gender swap transformation accuracy | ||
| 98.7% | Mara pronouns preserved (coreference test) | ||
| 100.0% | Setting and description preserved |