Data extraction

Extract key details from a given block of text.

Future event time

0-shot UtilityLogic
Model Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10 Total
Claude 3.5 Haiku100%100%100%100%100%100%100%100%100%100%100%
o4 Mini100%100%100%100%100%100%100%100%100%100%100%
o4 Mini High100%100%100%100%100%100%100%100%100%100%100%
Gemini 2.5 Pro100%100%100%100%100%100%100%100%100%100%100%
Gemini 2.5 Flash100%100%100%100%100%100%100%100%100%100%100%
Gemini 2.5 Flash Lite100%100%100%100%100%100%100%100%100%100%100%
Gemini 3 Pro (Preview)100%100%100%100%100%100%100%100%100%100%100%
Gemini 3 Flash (Preview)100%100%100%100%100%100%100%100%100%100%100%
Z.AI GLM 4.6100%100%100%100%100%100%100%100%100%100%100%
Z.AI GLM 4.7100%100%100%100%100%100%100%100%100%100%100%
Z.AI GLM 4.7 Flash100%100%100%100%100%100%100%100%100%100%100%
MoonshotAI: Kimi K2.5100%100%100%100%100%100%100%100%100%100%100%
GPT-4.1 Nano100%100%100%100%100%100%100%100%100%50%95%
Gemini Flash 1.5100%100%100%100%100%100%100%50%50%50%85%
Lumimaid v0.2 8B100%100%100%100%100%100%100%50%50%0%80%
Claude 3.5 Sonnet (new)100%100%100%100%100%100%50%50%50%50%80%
Cohere Command R+ (Aug. 2024)100%100%100%100%100%50%50%50%50%50%75%
Z.AI GLM 4.5100%100%100%100%100%50%50%50%50%50%75%
Phi-3 Mini 128k100%100%100%100%50%50%50%50%50%50%70%
Llama 3.1 70B100%100%50%50%50%50%50%50%50%50%60%
Gemma 2 27B100%50%50%50%50%50%50%50%50%50%55%
Llama 3.1 Euryale 70B v2.2100%50%50%50%50%50%50%50%50%50%55%
Mistral NeMO100%50%50%50%50%50%50%50%50%50%55%
Llama 3.1 405B100%50%50%50%50%50%50%50%50%50%55%
Llama 3.2 90B (Vision)100%50%50%50%50%50%50%50%50%50%55%
Rocinante 12B100%50%50%50%50%50%50%50%50%50%55%
Llama 3 TenyxChat-DaybreakStorywriter 70B100%50%50%50%50%50%50%50%50%50%55%
Claude 3.5 Sonnet50%50%50%50%50%50%50%50%50%50%50%
Claude 3.7 Sonnet50%50%50%50%50%50%50%50%50%50%50%
Claude 3 Haiku50%50%50%50%50%50%50%50%50%50%50%
Magnum 72B50%50%50%50%50%50%50%50%50%50%50%
Gemma 2 9B50%50%50%50%50%50%50%50%50%50%50%
Phi-3 Medium 128k50%50%50%50%50%50%50%50%50%50%50%
Phi-3.5 Mini 128k50%50%50%50%50%50%50%50%50%50%50%
Hermes 2 Theta 8B50%50%50%50%50%50%50%50%50%50%50%
Hermes 3 70B50%50%50%50%50%50%50%50%50%50%50%
Hermes 3 405B50%50%50%50%50%50%50%50%50%50%50%
WizardLM 2 8x22b50%50%50%50%50%50%50%50%50%50%50%
Cohere Command R+ (Apr. 2024)50%50%50%50%50%50%50%50%50%50%50%
Mistral Large50%50%50%50%50%50%50%50%50%50%50%
Mistral Large 250%50%50%50%50%50%50%50%50%50%50%
Ministral 3B50%50%50%50%50%50%50%50%50%50%50%
Ministral 8B50%50%50%50%50%50%50%50%50%50%50%
Mistral Medium50%50%50%50%50%50%50%50%50%50%50%
GPT-4o, May 13th (temp=0)50%50%50%50%50%50%50%50%50%50%50%
GPT-4o, May 13th (temp=1)50%50%50%50%50%50%50%50%50%50%50%
GPT-4o, Aug. 6th (temp=0)50%50%50%50%50%50%50%50%50%50%50%
GPT-4o, Aug. 6th (temp=1)50%50%50%50%50%50%50%50%50%50%50%
Llama 3 Euryale 70B v2.150%50%50%50%50%50%50%50%50%50%50%
Qwen 2 72B50%50%50%50%50%50%50%50%50%50%50%
Qwen 2.5 72B50%50%50%50%50%50%50%50%50%50%50%
Qwen 2 7B50%50%50%50%50%50%50%50%50%50%50%
lzlv 70B50%50%50%50%50%50%50%50%50%50%50%
MythoMist 7B50%50%50%50%50%50%50%50%50%50%50%
DeepSeek-V2 Chat50%50%50%50%50%50%50%50%50%50%50%
Claude 2.150%50%50%50%50%50%50%50%50%50%50%
Claude 3.0 Sonnet50%50%50%50%50%50%50%50%50%50%50%
Goliath 120B50%50%50%50%50%50%50%50%50%50%50%
GPT-4 Turbo50%50%50%50%50%50%50%50%50%50%50%
GPT-4o Mini (temp=0)50%50%50%50%50%50%50%50%50%50%50%
GPT-4o Mini (temp=1)50%50%50%50%50%50%50%50%50%50%50%
Llama 3 70B50%50%50%50%50%50%50%50%50%50%50%
Fimbulvetr 11B v250%50%50%50%50%50%50%50%50%50%50%
Llama 3.1 8B50%50%50%50%50%50%50%50%50%50%50%
Llama 3.2 3B50%50%50%50%50%50%50%50%50%50%50%
AI21 Jamba 1.5 Mini50%50%50%50%50%50%50%50%50%50%50%
AI21 Jamba 1.5 Large50%50%50%50%50%50%50%50%50%50%50%
EVA Qwen 2.5 14B100%50%50%50%50%50%50%50%50%0%50%
Magnum v2 72B50%50%50%50%50%50%50%50%50%50%50%
Inflection 3 (Productivity)50%50%50%50%50%50%50%50%50%50%50%
Inflection 3 (PI)50%50%50%50%50%50%50%50%50%50%50%
Llama 3.1 Nemotron 70B50%50%50%50%50%50%50%50%50%50%50%
Sao10K L3.1 70B Hanami x150%50%50%50%50%50%50%50%50%50%50%
MN GRAND Gutenberg Lyra4 12B Madness50%50%50%50%50%50%50%50%50%50%50%
GPT-4.150%50%50%50%50%50%50%50%50%50%50%
GPT-4.1 Mini50%50%50%50%50%50%50%50%50%50%50%
Claude Sonnet 450%50%50%50%50%50%50%50%50%50%50%
Claude Opus 450%50%50%50%50%50%50%50%50%50%50%
Claude Haiku 4.550%50%50%50%50%50%50%50%50%50%50%
Claude Sonnet 4.550%50%50%50%50%50%50%50%50%50%50%
Claude Opus 4.550%50%50%50%50%50%50%50%50%50%50%
Claude Opus 4.650%50%50%50%50%50%50%50%50%50%50%
Mistral Small Creative50%50%50%50%50%50%50%50%50%50%50%
Writer: Palmyra X550%50%50%50%50%50%50%50%50%50%50%
Gemini Pro 1.550%50%50%50%50%50%50%50%50%0%45%
AI21 Jamba50%50%50%50%50%50%50%50%50%0%45%
MythoMax 13B50%50%50%50%50%50%50%50%50%0%45%
Mistral Nemo 12B Celeste50%50%50%50%50%50%50%50%50%0%45%
Toppy M 7B50%50%50%50%50%50%50%50%0%0%40%
Llama 3.2 1B50%50%50%50%50%50%50%50%0%0%40%
Llama 3.2 11B (Vision)50%50%50%50%50%50%50%0%0%0%35%
Liquid: LFM 40B MoE50%50%50%50%50%50%50%0%0%0%35%
Claude 2.00%0%0%0%0%0%0%0%0%0%0%
57.90%