Write N of X

Write exactly N words/sentences/paragraphs...

200 word summary

0-shot Rule following
Model Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10 Total
Z.AI GLM 4.7100%100%98%98%98%98%98%98%98%92%98%
Gemini 3 Pro (Preview)100%100%100%100%98%98%92%92%92%77%95%
Gemini 3 Flash (Preview)100%100%100%100%98%92%92%77%77%77%91%
MoonshotAI: Kimi K2.598%98%98%98%98%92%77%77%54%27%82%
o4 Mini High100%100%100%98%98%98%77%54%54%9%79%
Z.AI GLM 4.7 Flash98%98%98%98%98%92%92%54%54%0%78%
o4 Mini100%100%100%100%98%92%54%54%27%9%73%
GPT-4o Mini (temp=1)100%100%100%100%92%77%77%54%27%2%73%
GPT-4o, Aug. 6th (temp=1)100%100%98%92%92%77%54%27%9%2%65%
Gemini 2.5 Pro100%100%98%77%77%77%54%27%0%0%61%
GPT-4o, Aug. 6th (temp=0)98%98%92%54%54%54%54%54%27%27%61%
GPT-4.1100%100%92%77%54%54%27%2%2%0%51%
GPT-4o, May 13th (temp=1)100%98%92%92%77%9%0%0%0%0%47%
Claude Sonnet 4.5100%98%77%77%54%27%0%0%0%0%43%
GPT-4o Mini (temp=0)98%98%92%9%9%9%9%9%9%9%35%
GPT-4.1 Nano100%92%27%27%9%0%0%0%0%0%26%
Magnum 72B100%92%0%0%0%0%0%0%0%0%19%
Writer: Palmyra X5100%77%2%2%0%0%0%0%0%0%18%
Claude Opus 492%77%0%0%0%0%0%0%0%0%17%
Cohere Command R+ (Aug. 2024)77%77%2%0%0%0%0%0%0%0%16%
Llama 3.1 Euryale 70B v2.292%54%0%0%0%0%0%0%0%0%15%
Claude Haiku 4.5100%27%0%0%0%0%0%0%0%0%13%
DeepSeek-V2 Chat98%27%0%0%0%0%0%0%0%0%13%
GPT-4o, May 13th (temp=0)54%27%27%0%0%0%0%0%0%0%11%
Llama 3.2 1B77%27%0%0%0%0%0%0%0%0%10%
Claude Opus 4.598%2%0%0%0%0%0%0%0%0%10%
Claude 2.1100%0%0%0%0%0%0%0%0%0%10%
Toppy M 7B100%0%0%0%0%0%0%0%0%0%10%
Phi-3 Mini 128k100%0%0%0%0%0%0%0%0%0%10%
MythoMist 7B100%0%0%0%0%0%0%0%0%0%10%
Sao10K L3.1 70B Hanami x192%0%0%0%0%0%0%0%0%0%9%
Claude Opus 4.677%0%0%0%0%0%0%0%0%0%8%
Mistral Small Creative77%0%0%0%0%0%0%0%0%0%8%
Qwen 2 72B77%0%0%0%0%0%0%0%0%0%8%
Claude Sonnet 477%0%0%0%0%0%0%0%0%0%8%
GPT-4 Turbo77%0%0%0%0%0%0%0%0%0%8%
MythoMax 13B54%0%0%0%0%0%0%0%0%0%5%
Llama 3.1 405B27%9%9%2%2%0%0%0%0%0%5%
Gemini 2.5 Flash Lite27%9%0%0%0%0%0%0%0%0%4%
Claude 3.7 Sonnet9%9%0%0%0%0%0%0%0%0%2%
Gemini 2.5 Flash9%2%2%0%0%0%0%0%0%0%1%
Magnum v2 72B9%0%0%0%0%0%0%0%0%0%1%
GPT-4.1 Mini0%0%0%0%0%0%0%0%0%0%0%
Claude 3.5 Sonnet (new)0%0%0%0%0%0%0%0%0%0%0%
Z.AI GLM 4.60%0%0%0%0%0%0%0%0%0%0%
Gemma 2 9B0%0%0%0%0%0%0%0%0%0%0%
Phi-3.5 Mini 128k0%0%0%0%0%0%0%0%0%0%0%
Llama 3.2 90B (Vision)0%0%0%0%0%0%0%0%0%0%0%
Claude 2.00%0%0%0%0%0%0%0%0%0%0%
Claude 3.5 Haiku0%0%0%0%0%0%0%0%0%0%0%
Lumimaid v0.2 8B0%0%0%0%0%0%0%0%0%0%0%
MN GRAND Gutenberg Lyra4 12B Madness0%0%0%0%0%0%0%0%0%0%0%
Hermes 2 Theta 8B0%0%0%0%0%0%0%0%0%0%0%
Llama 3 Euryale 70B v2.10%0%0%0%0%0%0%0%0%0%0%
AI21 Jamba0%0%0%0%0%0%0%0%0%0%0%
Phi-3 Medium 128k0%0%0%0%0%0%0%0%0%0%0%
Llama 3.2 3B0%0%0%0%0%0%0%0%0%0%0%
Qwen 2.5 72B0%0%0%0%0%0%0%0%0%0%0%
Claude 3 Haiku0%0%0%0%0%0%0%0%0%0%0%
AI21 Jamba 1.5 Mini0%0%0%0%0%0%0%0%0%0%0%
Rocinante 12B0%0%0%0%0%0%0%0%0%0%0%
Llama 3 70B0%0%0%0%0%0%0%0%0%0%0%
EVA Qwen 2.5 14B0%0%0%0%0%0%0%0%0%0%0%
Qwen 2 7B0%0%0%0%0%0%0%0%0%0%0%
Llama 3 TenyxChat-DaybreakStorywriter 70B0%0%0%0%0%0%0%0%0%0%0%
Claude 3.5 Sonnet0%0%0%0%0%0%0%0%0%0%0%
Gemini Flash 1.50%0%0%0%0%0%0%0%0%0%0%
Gemini Pro 1.50%0%0%0%0%0%0%0%0%0%0%
Gemma 2 27B0%0%0%0%0%0%0%0%0%0%0%
Hermes 3 70B0%0%0%0%0%0%0%0%0%0%0%
Hermes 3 405B0%0%0%0%0%0%0%0%0%0%0%
WizardLM 2 8x22b0%0%0%0%0%0%0%0%0%0%0%
Cohere Command R+ (Apr. 2024)0%0%0%0%0%0%0%0%0%0%0%
Mistral Large0%0%0%0%0%0%0%0%0%0%0%
Mistral Large 20%0%0%0%0%0%0%0%0%0%0%
Ministral 3B0%0%0%0%0%0%0%0%0%0%0%
Ministral 8B0%0%0%0%0%0%0%0%0%0%0%
Mistral Medium0%0%0%0%0%0%0%0%0%0%0%
lzlv 70B0%0%0%0%0%0%0%0%0%0%0%
Claude 3.0 Sonnet0%0%0%0%0%0%0%0%0%0%0%
Goliath 120B0%0%0%0%0%0%0%0%0%0%0%
Mistral NeMO0%0%0%0%0%0%0%0%0%0%0%
Fimbulvetr 11B v20%0%0%0%0%0%0%0%0%0%0%
Llama 3.1 8B0%0%0%0%0%0%0%0%0%0%0%
Llama 3.1 70B0%0%0%0%0%0%0%0%0%0%0%
Llama 3.2 11B (Vision)0%0%0%0%0%0%0%0%0%0%0%
Mistral Nemo 12B Celeste0%0%0%0%0%0%0%0%0%0%0%
AI21 Jamba 1.5 Large0%0%0%0%0%0%0%0%0%0%0%
Liquid: LFM 40B MoE0%0%0%0%0%0%0%0%0%0%0%
Inflection 3 (Productivity)0%0%0%0%0%0%0%0%0%0%0%
Inflection 3 (PI)0%0%0%0%0%0%0%0%0%0%0%
Llama 3.1 Nemotron 70B0%0%0%0%0%0%0%0%0%0%0%
Z.AI GLM 4.50%0%0%0%0%0%0%0%0%0%0%
14.05%