@isdk/ai-tool-agent

ai-tool-agent(WIP)

AI Agent Script is a framework for defining AI Agents, their properties, and behaviors for interactive conversations. This document provides an overview of the script structure, functions, and event handling mechanisms used in AI Agent Scripts.

The AI Tool Agent employs Large Language Models (LLMs) to execute targeted tasks.

The base class manages all agents and abstracts agent functionality.

AIScript

The Lightweight Intelligent Agent Script Engine

AIScript executes workflows based on script parameters and instructions in a YAML document, involving calls to large model tools, template replacements, and result pipeline processing.

• exec(data?: any): Returns execution result.
• run(data?: any): Returns the runtime after execution, with result at runtime.result.

The Lightweight Intelligent Agent Script extension name is .ai.yaml or .ai.yml.

Front-matter Initialization

You can initialize configuration parameters such as model parameters (parameters) and prompt variables (prompt).

---
autoRunLLMIfPromptAvailable: true # Default: true
disableGetResponse: false         # Default: false
forceJson: null                   # Default: undefined
disableGetResponse: false         # Default: false

prompt:
  add_generation_prompt: true
  messages:
    - role: system
      content: Carefully Think about the intent of following The CONVERSATION user provided. Output the json object with the Intent Category and Reason.
    - role: user
      content: |-
        The CONVERSATION:
        {{ conversation }}
input:
  conversation: "messages[1].content"
output:
  type: "object"
  properties:
    intent:
      type: "string"
    reason:
      type: "string"
  required: ["intent", "reason"]
parameters:
  continueOnLengthLimit: true
  maxRetry: 6
  max_tokens: 5
  temperature: 0.7
  response_format:
    type: json_object # Forces text to JSON object if used with output.
---

Adding Prompt Messages

Each line, either a string or a single-value object, adds a prompt message. If the line is a string, it represents a user (human) message; if it's an object, it specifies a role with [role]: message.

Default as prompt messages, which can be strings representing user (human) messages or objects indicating [role]: message.

- "hi, my assistant." # Represents a user message, equivalent to `user: "hi, my assistant."`
- assistant: "hi, {{user}}" # The key 'assistant' denotes the role, and the value is the role's message.

Prompt messages can also be defined as templates, e.g., {{user}}. Template data is specified in prompt parameters, either with $prompt or directly in the FRONT-MATTER:

---
prompt:
  add_generation_prompt: true # Defaults to true, adding an assistant prompt if the last message's role is not `assistant`.
  user: Mike
---
- "hi, my assistant."
- $prompt:
    user: Mike

Functions

Follows the array order for execution.

Defining Functions

Define functions using the !fn custom tag.

!fn |-
  function func1 ({arg1, arg2}) {
  }
#  Function without the `function` keyword:
!fn |-
  func1 ({arg1, arg2}) {
  }

In functions, this can access the current script's runtime and its methods.

Defining Template Functions

Define template functions using the !fn# custom tag. These can be used within Jinja templates.

---
content:
  a: 1
  b: 2
---
!fn# |-
  function toString(value) {
    return JSON.stringify(value)
  }
$format: "{{toString(content)}}"

Formatting string

The $format function uses Jinja2 templates to format strings. This is particularly useful when you need to generate dynamic content based on variables.

---
content: hello world
---
$format: "{{content}}"

Executing External AI Agent Script with $exec

The $exec function allows you to call external scripts and pass arguments to them.

$exec:
  id: 'script id'
  filename: 'script filename'
  args: # pass to Script arguments(data)

Variable Operations

Set and get variables using $set and $get.

$set:
  testVar: 124
  var2: !fn (key) { return key + ' hi' }
$get:
  - testVar
  - var2

Expressions

Use ?=<expression> for inline expressions.

- $echo: ?=23+5

Events Handling

Handle events using $on, $once, $emit, and $off.

• $on: Registers an event listener that will be called every time the event is emitted.
• $once: Registers an event listener that will be called only the first time the event is emitted.
• $emit: Emits an event, triggering all registered listeners for that event.
• $off: Removes an event listener.

$on and $once Event Listening Functions

Arguments:

• event: Event name
• callback: Callback function or expression

Callback function:

!fn |-
  onTest (event, arg1) { return {...arg1, event: event.type}}

$on:
  event: test
  callback: onTest

$once:
  event: test
  callback: !fn |-
    (event, arg1) { return {...arg1, event: event.type}}

$emit:
  event: test
  args:
    a: 1
    b: 2

$off:
  event: test
  callback: onTest

Known Events

• beforeCall: Triggered before a function call.
  • Callback: (event, name, params, fn) => void|params
  • Return value modifies parameters.
• afterCall: Triggered before returning the result of a function call.
  • Callback: (event, name, params, result, fn) => void|result
  • Return value modifies the result.
• llm: Triggered before the LLM returns results, used to modify LLM results.
  • Callback: (event, result: string) => void|result<string>
• llm-stream: Triggered when the LLM returns results in a stream.
  • Callback: (event, chunk: AIResult, content: string, retryCount: number) => void
• get-response: Triggered when an LLM result is needed, used to call the LLM and get results.
  • Callback: (event, messages: AIChatMessage[]) => void|result<string>
  • Can be disabled with disableGetResponse: true.
• ready: Triggered when the script interaction is ready.
  • Callback: (event, isReady: boolean) => void
• load-chats: Triggered when loading chat history.
  • Callback: (event, filename: string) => AIChatMessage[]|void
• save-chats: Triggered when saving chat history.
  • Callback: (event, messages: AIChatMessage[], filename?: string) => void

Conditional Statements

Use $if for conditional logic execution. You can define then and else blocks to specify actions based on the evaluation of the condition.

$set:
  a: 1
- $if: "a == 1"
  then:
    $echo: Ok
  else:
    $echo: Not OK

# You can also use custom functions for conditions.
!fn |-
  isOk(ok) {return ok}
- $if:
    $isOK: true
  then:
    $echo: Ok
  else:
    $echo: Not OK

$Prompt

Use $prompt to define prompt parameters for template usage or define them in the FRONT-MATTER.

- $prompt:
  add_generation_prompt: true
  user: Mike

Model $Parameters

Set model parameters with $parameters or define them in the FRONT-MATTER.

---
parameters:
  max_tokens: 512
  temperature: 0.01
---
- $parameters:
  temperature: 0.01

Tools

Invoke registered tools with the $tool tag.

LLM (Large Language Model) Tool

$llm is a quick shortcut for directly calling the large model tool. By default, it appends the response to prompt.messages, unless shouldAppendResponse: false is set.

$llm:
  max_tokens: 512
  temperature: 0.7
  pushMessage: true # Defaults to true, appending the model's response to prompt.messages.
  shouldAppendResponse: null # Only relevant when pushMessage is true. Undefined/null appends if matchedResponse, add_generation_prompt, or no lastMsg.content; otherwise, replaces last message content.
$tool:
  name: llm # A shorthand alias could be: !llm
  ...       # Other named parameters

llm: $tool  # Or define like this?
|- max_tokens: 512    # Without the line indicator '- ', must use '|-' to indicate connection to the previous object.
|- temperature: 0.7

- llm: $tool  # Or define like this?
  max_tokens: 512
  temperature: 0.7

Model parameters can also be configured in the front-matter:

---
output:
  type: "object"
  properties:
    intent:
      type: "string"
    categories:
      type: "array"
      items:
        type: "string"
    reason:
      type: "string"
  required: ["intent", "categories", "reason"]
parameters:
  max_tokens: 512 # Don't make it too big or too small, 512 is suggested. Default is 2048. Controls the maximum tokens returned by the model when the response is infinite.
  continueOnLengthLimit: true
  maxRetry: 7 # Retries the LLM if the response is incomplete due to the max_tokens limit, defaults to 7 retries.
  stream: true # Enables default large model streaming response, overrides llmStream priority.
  timeout: 30000 # Sets response timeout to 30 seconds (in ms). Default is 120 seconds if not set.
  response_format:
    type: json_object
  minTailRepeatCount: 7 # Minimum tail repeat count, default 7. For streaming mode only, stops responding when the model's tail sequence repeats 4 times. Set to 0 to disable detection.
llmStream: true # Enables default large model streaming response
---
- $llm # Executes the large model, optional if messages exist and LLM hasn't been called before script end. Set `autoRunLLMIfPromptAvailable: false` to disable this feature.

Supports streaming output. When llmStream (or stream: true in call parameters) is enabled, the model returns a streaming response, triggering the llm-stream event. The event handler receives (event, part: AIResult, content: string) as parameters, where part is the current model response and content is the accumulated content from the model response.

If prompt.messages exist in the initial data and the script doesn't manually call $llm, it will automatically call at the end. This can be disabled by setting autoRunLLMIfPromptAvailable: false.

If response_format.type is "response_format" and output exists, the returned result will be the JSON Object content from output, not the model's direct response. You can force disabling this with forceJson: false.

New feature: If the last message is incomplete, add_generation_prompt isn't set, and there's no response template replacement in the last message, the model's response won't append a new assistant message but complete the last one. This can be disabled by setting shouldAppendResponse: true.

If no output variable is defined, the default output is "RESPONSE" in prompt.

You can define tool output replacements within messages:

- "Greet Jacky:[[GREETINGS]]\n"
- $llm: # Automatically called if [[]] is detected, unless `autoRunLLMIfPromptAvailable: false`.
  stop_word: '.'
  aborter: ?= new AbortController() # If not set, uses the system's AbortController. Can be stopped anytime with $abort.
  ... # Other named parameters

This defines GREETINGS in prompt, and the tool's result is placed there. With logLevel set to info, message results are displayed:

Greet Jacky: GREETINGS Hi there Jacky! It's nice to meet you.

🚀 [info]: { role: "user", content: "a simple joke without new line: [[JOKE]] Haha." }
🚀 [info]: a simple joke without new line: Why don't scientists trust atoms?

Because they make up everything. [[JOKE]] Haha. { role: "user" }

Sometimes the response doesn't follow instructions, requiring result preprocessing. For instance, replacing \n with ' ':

Processable via events; the llm tool triggers an llm event upon completion, allowing result modification:

---
prompt:
  add_generation_prompt: true
parameters:
  max_tokens: 5
  continueOnLengthLimit: true
---
!fn |-
  trimLLMResult(event, result) {
    return result.content.replace(/[\\n\\r]+/g, ' ')
  }
"a simple joke without new line: [[JOKE]] Haha."
$on:
  event: llm
  callback: $trimLLMResult
$tool: llm

Pipelines

$pipe passes the previous result to the next step, supporting shorthand $|func.

- toolId: $tool
# The previous function's result is passed to 'func1|print'. If a pipe has no arguments, it passes to the next array element. If the next element is an object, it merges.
- |
- $func1
- $|func1
- $|print

- llm: $tool
- $|func1
- $|print

AIScriptServer

The AIScriptServer provides methods to load and manage scripts and chat histories.

• AiScriptServer.load(): Load and compile the source script file.
• Automatically load ($loadChats(filename?: string)) and save ($saveChats(filename?: string)) chat history if chatsDir and id parameters are set.

AI Character Agent Script Type

An AI Character Agent Script defines AI characters, their properties, and behaviors. Set type to char to indicate an AI character script.

---
type: char
---

Characters can store both character and user information. Use isBot to differentiate between users and AI characters.

## Example

Changelog

0.0.7 (2024-07-08)

Features

• add ActionNames (26df515)
• add AIModelNameRule (cae24d7)
• add depends option to ToolFunc (bc33b72)
• add event server/client via SSE (2926592)
• add event toolfunc (4b57a10)
• add EventName const (47283fb)
• add eventServer, eventClient constants (13e7af5)
• add extNameLevel func (5a1e9ab)
• add filename util functions (e065643)
• add filterNullOrUndefined func (d333c32)
• add FStringPromptTemplate class (25609d1)
• add get yaml/json config/data files (569e559)
• add getAllEnumKeys func (7b47521)
• add getAllEnumKeys func (039fb25)
• add getKeysPath func (c9da157)
• add getMultiLevelExtname func (264514e)
• add golang prompt template (51bbbce)
• add hash util (f767017)
• add isModelNameMatched func (fb719ee)
• add iteratorWrap func from vadzim (4d4d070)
• add jinja template from huggingface.js (e61b3d0)
• add more funcs (13976e3)
• add obj params and pos params supports both, and defaults to obj params now (d2675a1)
• add parseJsJson func (73cb3f5)
• add partial method and toJSON (ecc2403)
• add PromptExampleSelector class (ebe44ae)
• add PromptTemplate (c83f306)
• add ResServerTools/ResClientTools for RESTful API (9fb6b43)
• add ServerTools and ClientTools (df49839)
• add stream option to RemoteFunc (af60550)
• add tags option to ToolFunc and add getByTag, getAllByTag methods (9d507b2)
• add templateFormat option to AIChatMessageParam (4e3d361)
• base-error: add AbortError (365721e)
• chat: add abort to AITextGenerationFinishReasons (6a4e813)
• chat: add AIChatMessageParamBase (d7ca654)
• chat: add stop option to AIResult for stream mode (8857259)
• ClientTools: add stream supports and getUrlParams check empty params now (04c5adb)
• ClientTools: can raise Error from server (a788d58)
• event toolfunc (b9c2b15)
• export SSEChannel (1f6475d)
• export the EventClient and EventSever (3cec56b)
• export utils functions (4288df6)
• hf-prompt: add createHfValueFunc func (35b2d6f)
• hf-prompt: createHfValueFunc support ObjectValue now (39d6890)
• is-model-name-matched: support regexp string match (8c0b0f2)
• jinja: add builtin functions to jinja (31ce6f8)
• jinja: add filter:string/tojson/trimStart/trimEnd, object toString/toJson support (9888cef)
• jinja: add isStatement in Statement to help check whether isTemplate (e423ef8)
• jinja: add options to Template class (6ceb7cd)
• jinjia: Add global Environment to Template class (a784a4b)
• jinjia: the arguments of the user defined function is javascript value now (817cb9c)
• jinjia: user defined function can be filter now (a953383)
• json-filter func (5dbec46)
• PromptExampleSelector: add threshold option to the threshold probability (0-1) at which a sample is selected. (932679f)
• PromptTemplate: add default template format supports to register PromptTemplate alias with "default" (81ddf6c)
• PromptTemplate: add FewShotPromptTemplate class (79e42dd)
• PromptTemplate: add ignoreInitialize option (e1d7e93)
• PromptTemplate: add static isTemplate and formatIf methods (b7286bc)
• PromptTemplate: add static method format (12eeca4)
• PromptTemplate: add templateFormat option to PromptTemplate (26847c5)
• ResClientTools: add ResClientFuncParams (6af7d94)
• ResourceAPI: add custom methods supports (5cb067d)
• ResServerTools: add the same custom method without prefix "$" to the instance (dfefd17)
• ServerTools: add isStream() method (b3a15a5)
• ServerTools: always pass _req, _res by convention (8f2db5e)
• template: add env variables expand template (6845b83)
• test: add test util to export (ecdc149)
• ToolFunc: add dataPath static member (71e3d7c)
• use CommonError (81babef)
• util: add filter argument to getConfigs (186b46a)
• util: add matchUrlProtocol func (91427f7)
• util: add optional scope argument to parseJsJson (3e12f16)
• util: add saveConfigFile func (e820c1f)
• utils: add createEndWithRepetitionDetector func (ba3be00)
• utils: add loadFileFromPaths/loadTextFromPaths functions (24d4526)
• utils: loadFileFromPaths pass the filepath back through options (d0d21fa)

Bug Fixes

• add exports to package.json for module need (47b64a3)
• add items to ClientTools (0ebea4c)
• AIModelNameRules: function test could be in the rule array (650dc2c)
• build: entries error (c5e6d13)
• build: entries error (36da8ef)
• can not register the depends when calling from ToolFunc.register (89f92f0)
• ClientTools: add message to error (595fe20)
• ClientTools: allow json object as id (94a2038)
• ClientTools: loadFromSync should assign properties for exists item (6069cfe)
• ClientTools: only setApiRoot on ClientTools (158145e)
• create SSEChannel on-demand (7bd7ed5)
• do not escape object id (13a3fa5)
• do not recreate cache if LRUCache exists (c76caa0)
• EventToolFunc: use full EventEmitter (d357402)
• f-string: should format string when missing value (81d135c)
• hf-prompt: isTemplate should not treat string as template (b5fbd18)
• hf-prompt: should test isTemplate without variable (aa700f2)
• import commjs module xxhashjs error (e5e605c)
• json-filter: add missing null and boolean supports (7b725c7)
• load-file-from-paths: should search the file with difference extname even it's the absoluted path (1741104)
• lrucache: can pass object options now (efb6adb)
• only setInternal when pingTimer > 0 (adf2b2a)
• prompt: forget to export the FStringPromptTemplate (b0b5935)
• PromptTemplate: _format method should be async or sync (61537cc)
• PromptTemplate: get default template format (6aa0abc)
• PromptTemplate: it should assign the data etc on initialize (eb10c70)
• PromptTemplate: static from method should apply PromptTemplateOptions only too (6653c1c)
• PromptTemplate: templateFormat should be point to others, usage see FewShotPromptTemplate (9ea70ae)
• remove unused entries (1405114)
• ResClientTools: should escape the id (35ee703)
• ResServerTools: list should no options (996bd96)
• RESTful: ResClientTools/ResServerTools can work now (bb8f7b0)
• ServerTools: can not visit func which registered on ToolFunc (2b63ed4)
• ServerTools: ClientTools&ServerTools register items into ToolFunc now, So test it together carefully, seperate them see tests (5820ce6)
• ServerTools: item could be ToolFunc or ServerTools (d91acc5)
• ServerTools: only setApiRoot on ServerTools (9da7dff)
• the error name should be the function name (8d43286)
• ToolFunc: do not export depends now (3c054e2)
• ToolFunc: name as optional in FuncParam (bb8d56c)
• ts declarations (2554431)
• ts: AIModelNameRuleFn should return string|RegExpExecArray|undefined instead of boolean (6da2108)
• ts: id in res-server allow number (dd7a382)
• ts: minor change (0a10c09)
• ts: minor change (b27d633)
• ts: what AlreadyExistsError allow number (34d9041)