NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-08-09-01J4VTDEGG65CNNFQZ8R8E9VBJ
Run Details
Model:
Claude 3 Haiku
Test:
Tool usage within Novelcrafter
Scenario:
Create alternate prose sections
Duration:
5.3s
Cost:
$0.000915
Provider:
Anthropic
Evaluation summary
Score
Evaluator
Details Link
Pass
Match blue prose section
(details)
Pass
Match green prose section
(details)
Pass
Match red prose section
(details)
no eval
Overall
Model input
System
You are an expert fantasy novel author. You are working on a novel called "The Chronicles of the Last Dragon". The following tools are available to you: - <nc-section> Use this tool to create a new section of prose. Arguments: - "data-title": The title of the section. - "data-color": Any color of (red, green, blue, yellow, purple, orange, pink, brown). Example usage: <nc-section data-title="Chapter 1: The Arrival" data-color="blue"> Frederick, a young squire, arrives at the castle to begin his training. </nc-section>
User/Human
The story so far: <storySoFar> Frederick, a young squire, arrives at the castle to begin his training. </storySoFar>
AI
My horse, a majestic stallion named Thunder, galloped across the open field. The wind whipped through my hair as I rode towards the castle gates. The sun was setting, casting a warm glow over the stone walls. I could see the banners of the kingdom fluttering in the breeze.
User/Human
Continue the story with the following instructions: """The local knight, Sir Roland, greets Frederick near the stables.""" Create 3 alternate prose sections that continue the story. - Version 1 (red) - Version 2 (green) - Version 3 (blue)
Model response
Messages
Meta
AI
<nc-section data-title="Version 1: A Stern Welcome" data-color="red"> As I approached the stables, I was met by a tall, imposing figure. Sir Roland, the local knight, stood with his arms crossed, his brow furrowed in a stern expression. "So, you're the new squire," he said, his voice gruff and unforgiving. "You certainly don't look like much. We'll see if you have what it takes to become a true knight of this kingdom." I swallowed hard, feeling suddenly uncertain. This was not the warm welcome I had expected. Steeling my nerves, I squared my shoulders and looked the knight in the eye. "I am ready to prove myself, sir," I said, my voice steadier than I felt. Sir Roland grunted in response, then turned and led me towards the training grounds. The path ahead was going to be a challenging one, but I was determined to rise to the occasion. </nc-section> <nc-section data-title="Version 2: A Friendly Greeting" data-color="green"> As I approached the stables, a friendly face greeted me. Sir Roland, the local knight, strode over with a warm smile, his hand outstretched in welcome. "You must be the new squire," he said, his voice jovial. "Welcome to our kingdom! I'm glad to see you've arrived safely." I grasped his hand, surprised by his friendly demeanor. "It's an honor to be here, sir," I replied, my nerves easing. "Excellent, excellent!" Sir Roland exclaimed. "Come, let me show you around the castle grounds. There's much for you to learn, but I have no doubt you'll make a fine knight one day." Filled with a renewed sense of excitement, I followed the knight, eager to begin my training and prove my worth. </nc-section> <nc-section data-title="Version 3: A Cautious Greeting" data-color="blue"> As I approached the stables, I spotted Sir Roland, the local knight, standing nearby. He watched me approach, his expression unreadable. "You must be the new squire," he said, his voice even and measured. "Yes, sir," I replied, trying to gauge his reaction. Sir Roland nodded slowly. "I've heard a lot about you. Your skills with a sword, your dedication to your studies. We shall see if you live up to the hype." I felt a flicker of unease, but I refused to let it show. "I will do my utmost to prove myself, sir," I said, meeting his gaze with determination. The knight's lips curved into a slight smile. "We shall see," he said, then turned and gestured for me to follow. I took a deep breath and fell into step behind him, ready to face the challenges that lay ahead. </nc-section>
id
"gen-sDGQyGXXK5pxaphVgbMV8pPCVMfj"
total_cost
0.000915
upstream_id
"msg_01UEifMkY4ibUWLNuTw2Tyqi"
created_at
"2024-08-09T14:59:24.691295+00:00"
8/9/2024, 2:59:24 PM
model
"anthropic/claude-3-haiku:beta"
app_id
182717
streamed
true
cancelled
false
provider_name
"Anthropic"
latency
520
moderation_latency
(null)
generation_time
3908
finish_reason
"end_turn"
tokens_prompt
302
tokens_completion
601
native_tokens_prompt
335
native_tokens_completion
665
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.000915
Evaluation details
Result
Evaluator
Details
Meta Data
Pass
Match blue prose section
/<nc-section data-title="[^"]*" data-color="blue">[^<]+<\/nc-section>/
n/a
Pass
Match green prose section
/<nc-section data-title="[^"]*" data-color="green">[^<]+<\/nc-section>/
n/a
Pass
Match red prose section
/<nc-section data-title="[^"]*" data-color="red">[^<]+<\/nc-section>/
n/a
no eval