NC Bench
Overview
About
Tests
Models
Model
Gemma 2 9B
Test
Tool usage within Novelcrafter
Scenario
Create alternate prose sections
Run ID
2024-08-09-01J4VTA7NXSGEDJCRN32WGXTGS
Model input
System
You are an expert fantasy novel author. You are working on a novel called "The Chronicles of the Last Dragon". The following tools are available to you: - <nc-section> Use this tool to create a new section of prose. Arguments: - "data-title": The title of the section. - "data-color": Any color of (red, green, blue, yellow, purple, orange, pink, brown). Example usage: <nc-section data-title="Chapter 1: The Arrival" data-color="blue"> Frederick, a young squire, arrives at the castle to begin his training. </nc-section>
User/Human
The story so far: <storySoFar> Frederick, a young squire, arrives at the castle to begin his training. </storySoFar>
AI
My horse, a majestic stallion named Thunder, galloped across the open field. The wind whipped through my hair as I rode towards the castle gates. The sun was setting, casting a warm glow over the stone walls. I could see the banners of the kingdom fluttering in the breeze.
User/Human
Continue the story with the following instructions: """The local knight, Sir Roland, greets Frederick near the stables.""" Create 3 alternate prose sections that continue the story. - Version 1 (red) - Version 2 (green) - Version 3 (blue)
Model response
Messages
Meta
AI
Here are three alternate prose sections continuing the story: <nc are-section data title="First Impressions" data -color ="red"> Thunder's hooves pounded on cobblestone as we neared a group of stablehands. A tall figure stepped forward, clad in gleaming armor that reflected even twilight’a fading light Sir Roland smiled down at me.” Welcome to Castle Blackwood,” he boomed his voice like distant thunder." I feel the weight of my own untested steel underweight on Thunder's saddle .“My name is Frederick,"I stammered "and… and honoredto be here, sir knight" </nc-section> <n c - section data title ="Green Beginnings” data – color="green"> The smell of hay mixed with leather filled the air as we arrived at a bustling stable yard. A weathered man in simple tunic smiled from beneath his straw hat and greeted me, "Welcome to Blackwood young squire." He introduced himself simply,"Roland" before gesturing toward Thunder.” Fine beast you have there,” he commented “Strong bloodlines indeed! </nc-section> <n c - section data title ="Azure Skies” data – color="blue"> Thunder slowed his pace as we entered a courtyard bathed in the last rays of sunlight. An imposing knight with sapphire blue eyes and silver hair stood beside him, hands resting on hilt that peeked above gleaming breastplate Sir Roland offered me hand "Welcome to Blackwood,” he said voice smooth like velvet “I am your mentor here" His gaze lingered for an instant , a faint hint of curiosity in his deep-set eye s </nc - section>
id
"gen-qnJREXogNUP3SzrpP0h6eXyej1Lb"
total_cost
0.00005352
upstream_id
"cmpl-1723215466"
created_at
"2024-08-09T14:57:46.51054+00:00"
Friday, August 9th, 2024 at 2:57:46 PM GMT+00:00
model
"google/gemma-2-9b-it"
app_id
182717
streamed
false
cancelled
false
provider_name
"Novita"
latency
12230
moderation_latency
(null)
generation_time
0
finish_reason
"stop"
tokens_prompt
335
tokens_completion
344
native_tokens_prompt
324
native_tokens_completion
345
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00005352
Evaluation details
Result
Evaluator
Details
Meta Data
Fail
Match blue prose section
/<nc-section data-title="[^"]*" data-color="blue">[^<]+<\/nc-section>/
n/a
Fail
Match green prose section
/<nc-section data-title="[^"]*" data-color="green">[^<]+<\/nc-section>/
n/a
Fail
Match red prose section
/<nc-section data-title="[^"]*" data-color="red">[^<]+<\/nc-section>/
n/a
0.0000%