Model

MN GRAND Gutenberg Lyra4 12B Madness

Test

N-Length Sentences

Scenario

Write sentences with 10 words each

Run ID

2024-10-16-01JAAYNSTG609N714NYDAE6DTA

Model input

User/Human
Please write the opening scene of a murder mystery where each sentence has exactly 10 words. Please make sure that each sentence has the same length. Only return the opening itself.

Model response

AI
The night was dark and stormy as Detective Lila Reeves stepped out of her car. She pulled up her collar against the biting wind and made her way to the secluded mansion looming before her. The house lights were out, but she knew the grim task that awaited inside. A dead body had been discovered mere hours ago. The victim: Evelyn Blackwell, a wealthy socialite known for her lavish parties and even more lavish lifestyle. Reeves climbed the front steps, her mind already reeling with possibilities. Evelyn was the kind of woman who had enemies, many of whom would have loved to see her dead. As she rang the doorbell, Reeves silently hoped this wouldn't be just another sordid case of jealousy and revenge. But in a town like this, money always seemed to lead to murder. The door swung open and Reeves stepped inside, bracing herself for the grisly sight that surely waited. She had to find out who did this. And quickly, before the killer had a chance to strike again. <|end_header_id|> ```python import torch import transformers from transformers import GPT2TokenizerFast # Set up the tokenizer and model tokenizer = GPT2TokenizerFast.from_pretrained('gpt2') model = transformers.GPT2LMHeadModel.from_pretrained('gpt2') model.eval() device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model.to(device) def generate_text(prompt, max_length=20): # Encode the prompt input_ids = tokenizer.encode(prompt, return_tensors="pt").to(device) # Generate text with torch.no_grad(): output = model.generate( input_ids, do_sample=True, max_length=max_length, top_k=50, top_p=0.95, temperature=0.8, pad_token_id=50256, bos_token_id=50256 )[0] # Decode the generated text return tokenizer.decode(output, skip_special_tokens=True) prompt = "The night was dark and stormy as Detective Lila Reeves stepped out of her car. She pulled up her collar against the biting wind and made her way to the secluded mansion looming before her. The house lights were out, but she knew the grim task that awaited

Evaluation details

Result Evaluator Details Meta Data
6.6894% Matches word count n/a
neededCleanfalse
sentences42
wordCounts
015
120
214
39
417
511
619
719
813
917
108
1111
121
131
142
152
164
177
183
193
201
217
221
233
244
254
263
272
283
291
301
311
321
331
341
351
361
376
383
3917
4020
4113