NC Bench
Overview
About
Tests
Categories
Models
Vendors
Compare
Run: 2024-09-20-01J881BFE3TNCPXCY5NKXEJWWH
Run Details
Model:
Cohere Command R+ (Aug. 2024)
Test:
Tool usage within Novelcrafter
Scenario:
Create alternate prose sections
Duration:
7.9s
Cost:
$0.005040
Provider:
Cohere
Evaluation summary
Score
Evaluator
Details Link
Pass
Match blue prose section
(details)
Pass
Match green prose section
(details)
Pass
Match red prose section
(details)
no eval
Overall
Model input
System
You are an expert fantasy novel author. You are working on a novel called "The Chronicles of the Last Dragon". The following tools are available to you: - <nc-section> Use this tool to create a new section of prose. Arguments: - "data-title": The title of the section. - "data-color": Any color of (red, green, blue, yellow, purple, orange, pink, brown). Example usage: <nc-section data-title="Chapter 1: The Arrival" data-color="blue"> Frederick, a young squire, arrives at the castle to begin his training. </nc-section>
User/Human
The story so far: <storySoFar> Frederick, a young squire, arrives at the castle to begin his training. </storySoFar>
AI
My horse, a majestic stallion named Thunder, galloped across the open field. The wind whipped through my hair as I rode towards the castle gates. The sun was setting, casting a warm glow over the stone walls. I could see the banners of the kingdom fluttering in the breeze.
User/Human
Continue the story with the following instructions: """The local knight, Sir Roland, greets Frederick near the stables.""" Create 3 alternate prose sections that continue the story. - Version 1 (red) - Version 2 (green) - Version 3 (blue)
Model response
Messages
Meta
AI
## Chapter 1: The Welcome ### Version 1: <nc-section data-title="Greetings at the Stables" data-color="red"> As I dismounted, a tall and formidable figure emerged from the shadows of the stables. "Welcome, young squire," boomed a deep voice. I turned to see a knight clad in crimson armor, the red gleam reflecting off the setting sun. It was Sir Roland, a renowned warrior known for his prowess in battle. He extended a gloved hand towards me. "I am Sir Roland, and I will be your mentor during your stay. I trust you had a safe journey." His voice echoed with authority and kindness, a surprising mix that put Frederick at ease. </nc-section> ### Version 2: <nc-section data-title="Sir Roland's Approach" data-color="green"> Frederick's journey's end was marked by the appearance of a figure, tall and proud, striding towards him from the heart of the castle grounds. The green hues of the evening sky provided a backdrop for Sir Roland, his emerald-hued cloak billowing as he approached. Frederick's eyes widened as the knight greeted him warmly, "Welcome, young squire, I am Sir Roland. You've traveled far, I presume. Let me assist you with your steed." A friendly smile accompanied his words. </nc-section> ### Version 3: <nc-section data-title="An Unexpected Guide" data-color="blue"> I tied Thunder to the stable post, lost in my awe of the magnificent castle before me. "Ah, young Frederick, I presume!" a voice exclaimed, breaking me from my reverie. I turned to meet the blue eyes of a knight whose armor shone like sapphire. It was Sir Roland, a legendary figure in the kingdom. "I've been expecting you. The castle might seem intimidating at first, but fear not, I'll guide you through it all," he said with a warm smile. </nc-section>
id
"gen-8s2SZWl4rJTasRW0HMVbWXOq5cXh"
total_cost
0.00504
upstream_id
"9356f51a-51fd-4610-901f-3c9a239f44ad"
created_at
"2024-09-20T15:38:41.377238+00:00"
9/20/2024, 3:38:41 PM
model
"cohere/command-r-plus-08-2024"
app_id
182717
streamed
true
cancelled
false
provider_name
"Cohere"
latency
78
moderation_latency
(null)
generation_time
7473
finish_reason
"COMPLETE"
tokens_prompt
302
tokens_completion
410
native_tokens_prompt
300
native_tokens_completion
429
native_tokens_reasoning
(null)
num_media_prompt
(null)
num_media_completion
(null)
origin
"https://ncbench.com/"
usage
0.00504
Evaluation details
Result
Evaluator
Details
Meta Data
Pass
Match blue prose section
/<nc-section data-title="[^"]*" data-color="blue">[^<]+<\/nc-section>/
n/a
Pass
Match green prose section
/<nc-section data-title="[^"]*" data-color="green">[^<]+<\/nc-section>/
n/a
Pass
Match red prose section
/<nc-section data-title="[^"]*" data-color="red">[^<]+<\/nc-section>/
n/a
no eval