@isdk/ai-tool-agent

ai-tool-agent(WIP)

AI Agent Script is a framework for defining AI Agents, their properties, and behaviors for interactive conversations. This document provides an overview of the script structure, functions, and event handling mechanisms used in AI Agent Scripts.

The AI Tool Agent employs Large Language Models (LLMs) to execute targeted tasks.

The base class manages all agents and abstracts agent functionality.

AIScript

The Lightweight Intelligent Agent Script Engine

AIScript executes workflows based on script parameters and instructions in a YAML document, involving calls to large model tools, template replacements, and result pipeline processing.

• exec(data?: any): Returns execution result.
• run(data?: any): Returns the runtime after execution, with result at runtime.result.

The Lightweight Intelligent Agent Script extension name is .ai.yaml or .ai.yml.

Front-matter Initialization

You can initialize configuration parameters such as model parameters (parameters) and prompt variables (prompt).

---
autoRunLLMIfPromptAvailable: true # Default: true
disableGetResponse: false         # Default: false
forceJson: null                   # Default: undefined
disableGetResponse: false         # Default: false

prompt:
  add_generation_prompt: true
  messages:
    - role: system
      content: Carefully Think about the intent of following The CONVERSATION user provided. Output the json object with the Intent Category and Reason.
    - role: user
      content: |-
        The CONVERSATION:
        {{ conversation }}
input:
  conversation: "messages[1].content"
output:
  type: "object"
  properties:
    intent:
      type: "string"
    reason:
      type: "string"
  required: ["intent", "reason"]
parameters:
  continueOnLengthLimit: true
  maxRetry: 6
  max_tokens: 5
  temperature: 0.7
  response_format:
    type: json_object # Forces text to JSON object if used with output.
---

Adding Prompt Messages

Each line, either a string or a single-value object, adds a prompt message. If the line is a string, it represents a user (human) message; if it's an object, it specifies a role with [role]: message.

Default as prompt messages, which can be strings representing user (human) messages or objects indicating [role]: message.

- "hi, my assistant." # Represents a user message, equivalent to `user: "hi, my assistant."`
- assistant: "hi, {{user}}" # The key 'assistant' denotes the role, and the value is the role's message.

Prompt messages can also be defined as templates, e.g., {{user}}. Template data is specified in prompt parameters, either with $prompt or directly in the FRONT-MATTER:

---
prompt:
  add_generation_prompt: true # Defaults to true, adding an assistant prompt if the last message's role is not `assistant`.
  user: Mike
---
- "hi, my assistant."
- $prompt:
    user: Mike

Functions

Follows the array order for execution.

Defining Functions

Define functions using the !fn custom tag.

!fn |-
  function func1 ({arg1, arg2}) {
  }
#  Function without the `function` keyword:
!fn |-
  func1 ({arg1, arg2}) {
  }

In functions, this can access the current script's runtime and its methods.

Defining Template Functions

Define template functions using the !fn# custom tag. These can be used within Jinja templates.

---
content:
  a: 1
  b: 2
---
!fn# |-
  function toString(value) {
    return JSON.stringify(value)
  }
$format: "{{toString(content)}}"

Formatting string

The $format function uses Jinja2 templates to format strings. This is particularly useful when you need to generate dynamic content based on variables.

---
content: hello world
---
$format: "{{content}}"

Executing External AI Agent Script with $exec

The $exec function allows you to call external scripts and pass arguments to them.

$exec:
  id: 'script id'
  filename: 'script filename'
  args: # pass to Script arguments(data)

Variable Operations

Set and get variables using $set and $get.

$set:
  testVar: 124
  var2: !fn (key) { return key + ' hi' }
$get:
  - testVar
  - var2

Expressions

Use ?=<expression> for inline expressions.

- $echo: ?=23+5

Events Handling

Handle events using $on, $once, $emit, and $off.

• $on: Registers an event listener that will be called every time the event is emitted.
• $once: Registers an event listener that will be called only the first time the event is emitted.
• $emit: Emits an event, triggering all registered listeners for that event.
• $off: Removes an event listener.

$on and $once Event Listening Functions

Arguments:

• event: Event name
• callback: Callback function or expression

Callback function:

!fn |-
  onTest (event, arg1) { return {...arg1, event: event.type}}

$on:
  event: test
  callback: onTest

$once:
  event: test
  callback: !fn |-
    (event, arg1) { return {...arg1, event: event.type}}

$emit:
  event: test
  args:
    a: 1
    b: 2

$off:
  event: test
  callback: onTest

Known Events

• beforeCall: Triggered before a function call.
  • Callback: (event, name, params, fn) => void|params
  • Return value modifies parameters.
• afterCall: Triggered before returning the result of a function call.
  • Callback: (event, name, params, result, fn) => void|result
  • Return value modifies the result.
• llm: Triggered before the LLM returns results, used to modify LLM results.
  • Callback: (event, result: string) => void|result<string>
• llm-stream: Triggered when the LLM returns results in a stream.
  • Callback: (event, chunk: AIResult, content: string, retryCount: number) => void
• get-response: Triggered when an LLM result is needed, used to call the LLM and get results.
  • Callback: (event, messages: AIChatMessage[]) => void|result<string>
  • Can be disabled with disableGetResponse: true.
• ready: Triggered when the script interaction is ready.
  • Callback: (event, isReady: boolean) => void
• load-chats: Triggered when loading chat history.
  • Callback: (event, filename: string) => AIChatMessage[]|void
• save-chats: Triggered when saving chat history.
  • Callback: (event, messages: AIChatMessage[], filename?: string) => void

Conditional Statements

Use $if for conditional logic execution. You can define then and else blocks to specify actions based on the evaluation of the condition.

$set:
  a: 1
- $if: "a == 1"
  then:
    $echo: Ok
  else:
    $echo: Not OK

# You can also use custom functions for conditions.
!fn |-
  isOk(ok) {return ok}
- $if:
    $isOK: true
  then:
    $echo: Ok
  else:
    $echo: Not OK

$Prompt

Use $prompt to define prompt parameters for template usage or define them in the FRONT-MATTER.

- $prompt:
  add_generation_prompt: true
  user: Mike

Model $Parameters

Set model parameters with $parameters or define them in the FRONT-MATTER.

---
parameters:
  max_tokens: 512
  temperature: 0.01
---
- $parameters:
  temperature: 0.01

Tools

Invoke registered tools with the $tool tag.

LLM (Large Language Model) Tool

$llm is a quick shortcut for directly calling the large model tool. By default, it appends the response to prompt.messages, unless shouldAppendResponse: false is set.

$llm:
  max_tokens: 512
  temperature: 0.7
  pushMessage: true # Defaults to true, appending the model's response to prompt.messages.
  shouldAppendResponse: null # Only relevant when pushMessage is true. Undefined/null appends if matchedResponse, add_generation_prompt, or no lastMsg.content; otherwise, replaces last message content.
$tool:
  name: llm # A shorthand alias could be: !llm
  ...       # Other named parameters

llm: $tool  # Or define like this?
|- max_tokens: 512    # Without the line indicator '- ', must use '|-' to indicate connection to the previous object.
|- temperature: 0.7

- llm: $tool  # Or define like this?
  max_tokens: 512
  temperature: 0.7

Model parameters can also be configured in the front-matter:

---
output:
  type: "object"
  properties:
    intent:
      type: "string"
    categories:
      type: "array"
      items:
        type: "string"
    reason:
      type: "string"
  required: ["intent", "categories", "reason"]
parameters:
  max_tokens: 512 # Don't make it too big or too small, 512 is suggested. Default is 2048. Controls the maximum tokens returned by the model when the response is infinite.
  continueOnLengthLimit: true
  maxRetry: 7 # Retries the LLM if the response is incomplete due to the max_tokens limit, defaults to 7 retries.
  stream: true # Enables default large model streaming response, overrides llmStream priority.
  timeout: 30000 # Sets response timeout to 30 seconds (in ms). Default is 120 seconds if not set.
  response_format:
    type: json_object
  minTailRepeatCount: 7 # Minimum tail repeat count, default 7. For streaming mode only, stops responding when the model's tail sequence repeats 4 times. Set to 0 to disable detection.
llmStream: true # Enables default large model streaming response
---
- $llm # Executes the large model, optional if messages exist and LLM hasn't been called before script end. Set `autoRunLLMIfPromptAvailable: false` to disable this feature.

Supports streaming output. When llmStream (or stream: true in call parameters) is enabled, the model returns a streaming response, triggering the llm-stream event. The event handler receives (event, part: AIResult, content: string) as parameters, where part is the current model response and content is the accumulated content from the model response.

If prompt.messages exist in the initial data and the script doesn't manually call $llm, it will automatically call at the end. This can be disabled by setting autoRunLLMIfPromptAvailable: false.

If response_format.type is "response_format" and output exists, the returned result will be the JSON Object content from output, not the model's direct response. You can force disabling this with forceJson: false.

New feature: If the last message is incomplete, add_generation_prompt isn't set, and there's no response template replacement in the last message, the model's response won't append a new assistant message but complete the last one. This can be disabled by setting shouldAppendResponse: true.

If no output variable is defined, the default output is "RESPONSE" in prompt.

You can define tool output replacements within messages:

- "Greet Jacky:[[GREETINGS]]\n"
- $llm: # Automatically called if [[]] is detected, unless `autoRunLLMIfPromptAvailable: false`.
  stop_word: '.'
  aborter: ?= new AbortController() # If not set, uses the system's AbortController. Can be stopped anytime with $abort.
  ... # Other named parameters

This defines GREETINGS in prompt, and the tool's result is placed there. With logLevel set to info, message results are displayed:

Greet Jacky: GREETINGS Hi there Jacky! It's nice to meet you.

🚀 [info]: { role: "user", content: "a simple joke without new line: [[JOKE]] Haha." }
🚀 [info]: a simple joke without new line: Why don't scientists trust atoms?

Because they make up everything. [[JOKE]] Haha. { role: "user" }

Sometimes the response doesn't follow instructions, requiring result preprocessing. For instance, replacing \n with ' ':

Processable via events; the llm tool triggers an llm event upon completion, allowing result modification:

---
prompt:
  add_generation_prompt: true
parameters:
  max_tokens: 5
  continueOnLengthLimit: true
---
!fn |-
  trimLLMResult(event, result) {
    return result.content.replace(/[\\n\\r]+/g, ' ')
  }
"a simple joke without new line: [[JOKE]] Haha."
$on:
  event: llm
  callback: $trimLLMResult
$tool: llm

Pipelines

$pipe passes the previous result to the next step, supporting shorthand $|func.

- toolId: $tool
# The previous function's result is passed to 'func1|print'. If a pipe has no arguments, it passes to the next array element. If the next element is an object, it merges.
- |
- $func1
- $|func1
- $|print

- llm: $tool
- $|func1
- $|print

AIScriptServer

The AIScriptServer provides methods to load and manage scripts and chat histories.

• AiScriptServer.load(): Load and compile the source script file.
• Automatically load ($loadChats(filename?: string)) and save ($saveChats(filename?: string)) chat history if chatsDir and id parameters are set.

AI Character Agent Script Type

An AI Character Agent Script defines AI characters, their properties, and behaviors. Set type to char to indicate an AI character script.

---
type: char
---

Characters can store both character and user information. Use isBot to differentiate between users and AI characters.

## Example

Changelog

0.0.8 (2024-08-11)

Features

• add async-semaphore class (554d3d3)
• add cancelable-ability (1b59eed)
• add convert-params-size func (f40be61)
• add datetime helper functions (a690b78)
• add exclude option to load-file-from-paths (584a625)
• add IntSet class (3891766)
• add isWebStream func (c97c2f9)
• add jsonToMarkdownStr func (f426592)
• add messagesToText func (60f82c0)
• add readFilenamesRecursiveSync func (a1c321a)
• add ToolFuncPackage type (d2bbd91)
• AITextGenerationFinishReasons: add taskId for stream chunk (b688b47)
• CancelableAbility: add optional generateAsyncTaskId, cleanMultiTaskAborter methods and apply chunk.taskId for object stream (a4d87fe)
• CancelableAbility: add ReadableStream and multi-tasks with maxTaskConcurrency supports (9399647)
• CancelableAbility: add throwRejected method to TaskAbortController (5d1c366)
• CancelableAbility: can pass an AbortController into task now (76b3398)
• export formatISO func (5deb35c)
• export parseDateFormat, toDate (4719167)
• export parseISO from date-fns (ce9e4de)
• IntSet: add static methods (bac2f14)
• IntSet: add valueOf, toJSON, toString (5ade81c)
• more http status code added (8065169)
• Semaphore: add .alreadyRejected flag to signal (5d3aea3)
• Semaphore: add AbortSignal supports (25f2e35)
• Semaphore: add DefaultAsyncSemaphoreCapacity constant (0ec9d3f)
• tool-func: add asyncFeatures option (b4ce16c)
• ToolFunc: add _asyncFeatures for ability injection (2d60803)
• ToolFunc: add static hasAsyncFeature (4715821)
• util: add paramsSizeToScaleStr,scaleStrToParamsSize funcs (89230c7)

Bug Fixes

• base-error: add default AbortError name for AbortError (635ea51)
• CancelableAbility: abort should assign data to reason if reason is an object (4d20bec)
• CancelableAbility: Because cannot standardize the internal variable format of terser, we have switched to managing it ourselves. (787e914)
• CancelableAbility: should abort waiting task immediately (82777c4)
• CancelableAbility: should clean task if task raise error (4a4ae0b)
• CancelableAbility: should hide parent, streamController props of TaskAbortController (bf79f04)
• CancelableAbility: verify passed aborter type (30a068d)
• jinja2: throw SyntaxError(`Unknown statement type: ${tokens[current].type} ${tokens[current].value} with value (fa7e86d)
• jinja: Pass undefined value as null value, do not throw error (7f1a703)
• readFilenamesRecursiveSync: should use absoluteDir to check whether visitedDirs (bf5c7a6)
• Semaphore: public maxConcurrency readonly (b2ddb88)
• ToolFunc: const ToolAsyncMultiTas should be ToolAsyncMultiTaskBit (f195536)
• typo (59bba56)
• util: jsonFilterToWhere should convert date to UTC ISO Date string (b995268)
• utils: filter duplicaion punctuations in split-sentence (7dc9f99)

Refactor

• CancelableAbility: keep abort compatibility (1365815)
• CancelableAbility: make abort more stable (b035b8f)
• CancelableAbility: new TaskAbortController Class with parent, streamController and abort (9be5ab4)
• CancelableAbility: the default cleanMultiTaskAborter (48ac131)
• createCallbacksTransformer more general (2f76138)
• end-with-repeated-sequence return matched now (0a94b7b)
• export createAbilityInjector (a5ceeb6)
• ToolFunc: remove async keyword to avoid more Promise wrapped (199ae62)