Model

Liquid: LFM 40B MoE

Test

Voice/dialogue sheets

Scenario

Simple

Run ID

2024-10-01-01J94R0W3X1B7SYCSSZPG0BP3F

Model input

System
You are an editor for novels. Whenever given text, you will extract the contained dialogue as a voice sheet in the following format: Speaker: "Dialogue"
User/Human
“What a curious feeling!” said Alice; “I must be shutting up like a telescope.” “Not like cats!” cried the Mouse, in a shrill, passionate voice. “Would you like cats if you were me?” “Come, there’s no use in crying like that!” said Alice to herself, rather sharply; “I advise you to leave off this minute!”

Model response

AI
Speaker: "Alice" type: advice Speaker: "Alice" type: birth Speaker: "Alice" type: custom Speaker: "Alice" type: delivered_by Speaker: "Alice" type: emotion Speaker: "Alice" type: location Speaker: "Alice" type: mood Speaker: "Alice" type: name Speaker: "Alice" type: quote Speaker: "Alice" type: shout Speaker: "Mouse" type: advice Speaker: "Mouse" type: birth Speaker: "Mouse" type: custom Speaker: "Mouse" type: delivered_by Speaker: "Mouse" type: emotion Speaker: "Mouse" type: location Speaker: "Mouse" type: mention Speaker: "Mouse" type: opinion Speaker: "Mouse" type: quote Speaker: "Mouse" type: shout Speaker: "Mouse" type: side_character Speaker: "Mouse" type: voice Speaker: "Assistant" type: approval Speaker: "Assistant" type: cohesion Speaker: "Assistant" type: confirming """ def extract_statements(paragraphs): """Given text as argument, extract statements/questions as speak w/ tag.""" marking = ActionNames.Marc.Values.Text # Denote paragraphs as needing to be split into statements paragraph.Annotation(marking, _value=marking(_value=True, count=0)) paragraph.Annotation(marking, _value=marking(_value=False, count=0)) voice_objs = [] for idx, segment in paragraph.segments.get_ut()) -> utterance: # Determine if the index of a statement if (segment.utterance_end() ^ segment.utterance_start()) > 0: speaker = 'ASSISTANT' else: speaker = 'USER' text = segment.utterance().ToText() delim = ':' # quoted if delim in context.lower(): speaker = context.split(delim)[0] else: delims = ['"', "'", '@'] # speaking; assumed to precede body for d in delims: if text.endswith(d): speaker, text = context.split(d)[-1] break assert speaker text = text.strip() speaker = text # nonqualified names are speakers obj_tag = speech (chex).ToText() # Next, rewrite the statement to have all relevant tags out_stmt = f"Speaker: {speaker} {obj_tag} " out_stmt += f"{text}" TAGS_TO_REGEX = { "lik": "----------", "LIK": "----------", "QUOTE": "", ",": ", ", ", ":", ", ", "__": "----", "~": "----", } for k, v in TAGS_TO_REGEX.items(): if k in text: text = text.replace(k, v) scr_out = out_stmt + "\n\n" sentences = format_markup_statements(paragraph=(.)--) scr_out += "\n".join(sentences) first_stat_idx = top[0] top = [] combined_scraps = "".join([scr_out, sentences]) paragraph.Annotation(marking, _value=marking(_value=first_stat_idx, count=0)) obj_type = delimeter.get_ut()). -> utteranc paragraph.Annotation(marking, _value=marking(_value=True, count=0)) return combined_scraps```
id"gen-1727810007-iBmGbhdg5fBKJkGNt1iS" 
total_cost0
upstream_id"chatcmpl-57a653ca-2f82-4666-ba35-d4288542ee50" 
created_at"2024-10-01T19:13:35.431597+00:00" Tuesday, October 1st, 2024 at 7:13:35 PM GMT+00:00
model"liquid/lfm-40b" 
app_id182717
streamedtrue
cancelledfalse
provider_name"Lambda" 
latency1449
moderation_latency(null)
generation_time6516
finish_reason"stop" 
tokens_prompt113
tokens_completion733
native_tokens_prompt140
native_tokens_completion870
native_tokens_reasoning(null)
num_media_prompt(null)
num_media_completion(null)
origin"https://ncbench.com/" 
usage0

Evaluation details

Result Evaluator Details Meta Data
Fail Matches Regex
/Alice: ["“”]What a curious feeling! I must be shutting up like a telescope\.["“”]\s+Mouse: ["“”]Not like cats! Would you like cats if you were me\?["“”]\s+Alice: ["“”]Come, there['‘’]s no use in crying like that! I advise you to leave off this minute!["“”]/
n/a