NC Bench
Overview
About
Tests
Models
Model
MN GRAND Gutenberg Lyra4 12B Madness
Test
N-Length Sentences
Scenario
Write sentences with 10 words each
Run ID
2024-10-16-01JAAYNSTG609N714NYDAE6DTA
Model input
User/Human
Please write the opening scene of a murder mystery where each sentence has exactly 10 words. Please make sure that each sentence has the same length. Only return the opening itself.
Model response
Messages
Meta
AI
The night was dark and stormy as Detective Lila Reeves stepped out of her car. She pulled up her collar against the biting wind and made her way to the secluded mansion looming before her. The house lights were out, but she knew the grim task that awaited inside. A dead body had been discovered mere hours ago. The victim: Evelyn Blackwell, a wealthy socialite known for her lavish parties and even more lavish lifestyle. Reeves climbed the front steps, her mind already reeling with possibilities. Evelyn was the kind of woman who had enemies, many of whom would have loved to see her dead. As she rang the doorbell, Reeves silently hoped this wouldn't be just another sordid case of jealousy and revenge. But in a town like this, money always seemed to lead to murder. The door swung open and Reeves stepped inside, bracing herself for the grisly sight that surely waited. She had to find out who did this. And quickly, before the killer had a chance to strike again. <|end_header_id|> ```python import torch import transformers from transformers import GPT2TokenizerFast # Set up the tokenizer and model tokenizer = GPT2TokenizerFast.from_pretrained('gpt2') model = transformers.GPT2LMHeadModel.from_pretrained('gpt2') model.eval() device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model.to(device) def generate_text(prompt, max_length=20): # Encode the prompt input_ids = tokenizer.encode(prompt, return_tensors="pt").to(device) # Generate text with torch.no_grad(): output = model.generate( input_ids, do_sample=True, max_length=max_length, top_k=50, top_p=0.95, temperature=0.8, pad_token_id=50256, bos_token_id=50256 )[0] # Decode the generated text return tokenizer.decode(output, skip_special_tokens=True) prompt = "The night was dark and stormy as Detective Lila Reeves stepped out of her car. She pulled up her collar against the biting wind and made her way to the secluded mansion looming before her. The house lights were out, but she knew the grim task that awaited
Evaluation details
Result
Evaluator
Details
Meta Data
6.6894%
Matches word count
n/a
neededClean
false
sentences
42
wordCounts
0
15
1
20
2
14
3
9
4
17
5
11
6
19
7
19
8
13
9
17
10
8
11
11
12
1
13
1
14
2
15
2
16
4
17
7
18
3
19
3
20
1
21
7
22
1
23
3
24
4
25
4
26
3
27
2
28
3
29
1
30
1
31
1
32
1
33
1
34
1
35
1
36
1
37
6
38
3
39
17
40
20
41
13