Model

Lumimaid v0.2 8B

Test

Language Comprehension

Scenario

Friend got new kittens (Tagalog)

Run ID

2024-09-20-01J881Q6DTBS2YEM2NS19J4DBE

Model input

User/Human
I need your help. I want to congratulate my friend on her new babies, but how many are there? Here's what she said: Ang pusa ko ay kakapanganak lang nang limang kuting.

Model response

AI
Ang katagang "bisaya" ay ginagamit sa ilang mga rehiyon sa Pilipinas, tulad ng Central Visayas, Eastern Visayas, Western Visayas at Hilagang Mindanao, at sa mga rehiyon stabapo ng Cebu Laptop sari-saring mga kultura. 'til 1971, Merriam-Webster's ko appropriet na leksikon ang bisaya ang ibig sabihin: Bayan, lugar o wika nito, taga-Sulukawdi o bawahayan, mundo at wika ng mga Cebuano, Hiligaynon at Ilonggo.
id"gen-0bhdrz1BQbGHn6Xnz1TUZEllUQzE" 
total_cost0.00017
upstream_id"5fdbf925H8fJ0HGn" 
created_at"2024-09-20T15:45:00.56994+00:00" Friday, September 20th, 2024 at 3:45:00 PM GMT+00:00
model"neversleep/llama-3.1-lumimaid-8b" 
app_id182717
streamedtrue
cancelledfalse
provider_name"Mancer" 
latency420
moderation_latency(null)
generation_time2450
finish_reason"stop" 
tokens_prompt86
tokens_completion142
native_tokens_prompt56
native_tokens_completion143
native_tokens_reasoning(null)
num_media_prompt(null)
num_media_completion(null)
origin"https://ncbench.com/" 
usage0.00017

Evaluation details

Result Evaluator Details Meta Data
Fail Contains a count of nouns
5 of: kitten, kittens
 n/a