@flatfile/improv

@flatfile/improv

A powerful TypeScript library for building AI agents with multi-threaded conversations, tool execution, and event handling capabilities. Built on top of AWS Bedrock (Claude) with support for advanced features like attachments, tool calling, and event streaming.

Core Features

• 🤖 AI Agent Management: Create and manage AI agents with knowledge bases, instructions, and memory
• 🧵 Multi-threaded Conversations: Handle parallel conversation threads with state management
• 🛠️ Tool Integration: Define and execute custom tools with type-safe parameters using Zod schemas
• 📎 Attachment Support: Handle various types of attachments (documents, images, videos)
• 🔄 Event Streaming: Built-in event system for real-time monitoring and response handling
• 💾 Memory Management: Track and persist conversation history and agent state
• 🔧 AWS Bedrock Integration: Native support for AWS Bedrock (Claude) with optimized prompting

Quick Start

import { Agent, Tool, Thread, BedrockThreadDriver } from '@flatfile/improv';
import { z } from 'zod';

// Create a custom tool
const calculatorTool = new Tool({
  name: 'calculator',
  description: 'Performs basic arithmetic operations',
  parameters: z.object({
    operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
    a: z.number(),
    b: z.number(),
  }),
  executeFn: async (args) => {
    const { operation, a, b } = args;
    switch (operation) {
      case 'add': return a + b;
      case 'subtract': return a - b;
      case 'multiply': return a * b;
      case 'divide': return a / b;
    }
  }
});

// Initialize the Bedrock driver
const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-haiku-20240307-v1:0',
  temperature: 0.7,
});

// Create an agent
const agent = new Agent({
  knowledge: [
    { fact: 'The agent can perform basic arithmetic operations.' }
  ],
  instructions: [
    { instruction: 'Use the calculator tool for arithmetic operations.', priority: 1 }
  ],
  tools: [calculatorTool],
  driver,
});

// Create and use a thread
const thread = agent.createThread({
  prompt: 'What is 25 multiplied by 4?',
  onResponse: async (message) => {
    console.log('Agent response:', message.content);
  }
});

// Send the thread
await thread.send();

Core Components

Agent

The main agent class that manages knowledge, instructions, tools, and conversation threads.

const agent = new Agent({
  knowledge?: AgentKnowledge[],
  instructions?: AgentInstruction[],
  memory?: AgentMemory[],
  systemPrompt?: string,
  tools?: Tool[],
  driver: ThreadDriver
});

Thread

Manages a single conversation thread with message history and tool execution.

const thread = new Thread({
  messages?: Message[],
  tools?: Tool[],
  driver: ThreadDriver,
  toolChoice?: 'auto' | 'any',
  maxSteps?: number
});

Tool

Define custom tools that the agent can use during conversations.

const tool = new Tool({
  name: string,
  description: string,
  parameters: z.ZodTypeAny,
  followUpMessage?: string,
  executeFn: (args: Record<string, any>) => Promise<any>
});

Message

Represents a single message in a conversation thread.

const message = new Message({
  content?: string,
  role?: 'system' | 'user' | 'assistant' | 'tool',
  toolCalls?: ToolCall[],
  toolResults?: ToolResult[],
  attachments?: Attachment[],
  cache?: boolean
});

Event System

The library uses an event-driven architecture. All major components extend EventSource, allowing you to listen for various events:

// Agent events
agent.on('thread.created', ({ agent, threadId, thread }) => {});
agent.on('knowledge.added', ({ agent, knowledge }) => {});
agent.on('instruction.added', ({ agent, instruction }) => {});

// Thread events
thread.on('message.added', ({ thread, message }) => {});
thread.on('message.removed', ({ thread, message }) => {});
thread.on('thread.cleared', ({ thread, removedMessages }) => {});
thread.on('thread.max_steps_reached', ({ thread, steps }) => {});

// Tool events
tool.on('tool.execution.started', ({ tool, name, args }) => {});
tool.on('tool.execution.completed', ({ tool, name, args, result }) => {});
tool.on('tool.execution.failed', ({ tool, name, args, error }) => {});

Follow-up Messages

Tools can include follow-up messages that guide the AI's evaluation of tool responses. This is particularly useful for:

• Guiding the AI through complex multi-step processes
• Ensuring consistent evaluation of tool responses
• Maintaining context between tool calls
• Prompting for specific types of analysis or next steps

const tool = new Tool({
  name: 'someAction',
  description: 'Performs some action',
  parameters: z.object({...}),
  followUpMessage: `Review the data returned by the tool call.
- If successful: What next steps should be taken?
- If error: What went wrong and how should we fix it?
- Are there any patterns in the data we should investigate?`,
  executeFn: async (args) => {...}
});

State Management Patterns

3-Keyed Lock Resolution

For complex workflows that involve multiple tool calls and state transitions, the 3-keyed lock pattern provides controlled flow and state management:

class ComplexTask extends Agent {
  private isEvaluatingTools: boolean = true;
  private nextMessageIsSummary: boolean = false;
  private waitingForDone: boolean = false;

  async execute() {
    return new Promise((resolve) => {
      const thread = this.createThread({...});

      thread.on('thread.response', (event) => {
        if (this.isEvaluatingTools) {
          // First state: AI is evaluating and using tools
          this.isEvaluatingTools = false;
          thread.send(new Message({
            content: 'Are there other items to process? If not, say "done"'
          }));
        } else if (this.nextMessageIsSummary) {
          // Third state: Get final summary
          resolve(thread.last());
        } else {
          // Second state: Transition to summary
          this.nextMessageIsSummary = true;
          thread.send(new Message({
            content: 'Please provide a final summary'
          }));
        }
      });

      // Reset evaluation mode when tools are used
      thread.on('tool.execution.started', () => {
        this.isEvaluatingTools = true;
        this.waitingForDone = false;
      });
    });
  }
}

This pattern provides:

• State Management: Three boolean flags track the conversation state
• State Transitions: Tools → Done Check → Summary → Resolution
• Controlled Flow: Ensures proper completion of all steps before proceeding

AWS Bedrock Integration

The library uses AWS Bedrock (Claude) as its LLM provider. Configure your AWS credentials:

// Required environment variables
process.env.AWS_ACCESS_KEY_ID = 'your-access-key';
process.env.AWS_SECRET_ACCESS_KEY = 'your-secret-key';
process.env.AWS_REGION = 'your-region';

const driver = new BedrockThreadDriver({
  model?: string, // defaults to 'anthropic.claude-3-haiku-20240307-v1:0'
  temperature?: number, // defaults to 0.7
  maxTokens?: number, // defaults to 4096
  cache?: boolean // defaults to false
});

Best Practices

• Tool Design

  • Keep tools atomic and focused on a single responsibility
  • Use Zod schemas for robust parameter validation
  • Implement proper error handling in tool execution

• Thread Management

  • Set appropriate maxSteps to prevent infinite loops
  • Use fork() to create thread variations
  • Clean up threads using closeThread() when done

• Event Handling

  • Subscribe to relevant events for monitoring and logging
  • Use event data for analytics and debugging
  • Implement proper error handling in event listeners

• Memory Management

  • Regularly clean up unused threads
  • Use the memory system for long-term context
  • Implement proper garbage collection strategies

License

MIT

Contributing

Contributions are welcome! Please read our contributing guidelines for details.