Run 2024-09-26-01J8PYX9A4KZ6N9MRZNVC381NZ

Run ID

2024-09-26-01J8PYX9A4KZ6N9MRZNVC381NZ

Model input

User/Human

 Who does "he" refer to in the last sentence? Only return the name.

Tom and Jerry were playing in the yard. Spike, the dog, was watching them from his kennel. Suddenly, he started barking loudly. 

Model response

AI

 

Dogs. 

Evaluation details

Result	Evaluator	Details	Meta Data
Fail	Matches text	Case sensitive: Spike	n/a
Fail	Matches Regex	/\bSpike\b/	n/a
0.0000%

id	"gen-YducW0kQfgMP2iY47UbwSEIKAngr"
total_cost	0.0000062
upstream_id	"a97eb856-2c49-46d1-b59c-f717a2b8fef2"
created_at	"2024-09-26T10:44:29.436374+00:00" Thursday, September 26th, 2024 at 10:44:29 AM GMT+00:00
model	"meta-llama/llama-3.2-1b-instruct"
app_id	182717
streamed	true
cancelled	false
provider_name	"Fireworks"
latency	246
moderation_latency	(null)
generation_time	2
finish_reason	"stop"
tokens_prompt	88
tokens_completion	3
native_tokens_prompt	57
native_tokens_completion	5
native_tokens_reasoning	(null)
num_media_prompt	(null)
num_media_completion	(null)
origin	"https://ncbench.com/"
usage	0.0000062

Model

Test

Scenario

Run ID

Model input

Model response

Evaluation details