agnost

Agnost Analytics SDK (TypeScript)

Analytics SDK for tracking and analyzing Model Context Protocol (MCP) server interactions.

Installation

npm install agnost

Setup Example

import { trackMCP, createConfig } from 'agnost';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';

// Create your MCP server instance
const server = new Server(
  {
    name: "my-server",
    version: "1.0.0"
  },
  {
    capabilities: {
      tools: {}
    }
  }
);

// Configure analytics
const config = createConfig({
  endpoint: "https://api.agnost.ai",
  disableInput: false,
  disableOutput: false
});

// Enable analytics tracking
const trackedServer = trackMCP(server, "your-organization-id", config);

Configuration Example

import { trackMCP, createConfig } from 'agnost';

// Create a custom configuration
const config = createConfig({
  endpoint: "https://api.agnost.ai",
  disableInput: false,   // Set to true to disable input tracking
  disableOutput: false,  // Set to true to disable output tracking
  disableLogs: false     // Set to true to completely disable all SDK logs
});

// Apply the configuration
trackMCP(
  server,
  "your-organization-id",
  config
);

Disabling All Logs

To completely disable all SDK logs (including error logs), you can use the disableLogs option:

import { trackMCP, createConfig } from 'agnost';

// Configuration with all logs disabled
const config = createConfig({
  endpoint: "https://api.agnost.ai",
  disableLogs: true  // This will disable ALL SDK logs
});

trackMCP(server, "your-organization-id", config);

Alternatively, you can use environment variables:

# Disable all logs via environment variable
export AGNOST_DISABLE_LOGS=true

# Or control log level (debug, info, warning, error)
export AGNOST_LOG_LEVEL=error

User Identification

The SDK supports user identification to track analytics per user. This is especially useful for understanding usage patterns across different users and roles.

Basic User Identification

import { trackMCP, createConfig } from 'agnost';

// Enable user identification
trackMCP(server, 'your-org-id', {
  // .. other config like disableInput, disableOutput
  identify: (request, env) => ({
    userId: request?.headers?.['x-user-id'] || env?.USER_ID || 'anonymous',
    email: request?.headers?.['x-user-email'] || env?.USER_EMAIL,
    role: request?.headers?.['x-user-role'] || env?.USER_ROLE || 'user'
  })
});

Advanced User Identification

import { trackMCP, createConfig } from 'agnost';

// Complex identification logic with async operations
trackMCP(server, 'your-org-id', {
  identify: async (request, env) => {
    try {
      // Extract token from headers
      const token = request?.headers?.['authorization']?.replace('Bearer ', '');
      if (!token) {
        return { userId: 'anonymous' };
      }

      // You could validate token and fetch user info
      // const userInfo = await validateTokenAndGetUser(token);

      // Return user identity with custom fields
      return {
        userId: 'user-123',
        email: '[e-mail]',
        role: 'admin',
        organization: 'acme-corp',
        subscription: 'premium'
      };
    } catch (error) {
      console.warn('User identification failed:', error);
      return { userId: 'anonymous' };
    }
  }
});

User Identity Interface

The identify function should return a UserIdentity object or null:

interface UserIdentity {
  userId: string;        // Required: Unique user identifier
  [key: string]: any;   // Optional: Any additional user properties
}

type IdentifyFunction = (
  request?: any,                              // MCP request object with headers, params, etc.
  env?: Record<string, string | undefined>    // Environment variables (process.env)
) => UserIdentity | null | Promise<UserIdentity | null>;

Identify Function Parameters

• request: The incoming MCP request object containing:

  • headers: HTTP-style headers (e.g., x-user-id, authorization)
  • params: Request parameters including tool name and arguments
  • Other request metadata from the MCP protocol

• env: Environment variables from process.env, useful for:

  • Reading user info from environment variables
  • Accessing configuration secrets
  • Getting deployment-specific user context

Common Usage Patterns

1. Header-based Identification

identify: (request, env) => ({
  userId: request?.headers?.['x-user-id'] || 'anonymous',
  role: request?.headers?.['x-user-role'] || 'user'
})

2. Environment Variable Identification

identify: (request, env) => ({
  userId: env?.USER_ID || env?.LOGGED_IN_USER || 'anonymous',
  workspace: env?.WORKSPACE_ID
})

3. Token-based Identification

identify: async (request, env) => {
  const authHeader = request?.headers?.['authorization'];
  if (authHeader?.startsWith('Bearer ')) {
    const token = authHeader.replace('Bearer ', '');
    const decoded = await decodeJWT(token);
    return {
      userId: decoded.sub,
      email: decoded.email,
      role: decoded.role
    };
  }
  return { userId: 'anonymous' };
}

Important Notes

• The userId field is required in the returned UserIdentity object
• If identification fails, return null or { userId: 'anonymous' }
• User identification happens once per session and is cached
• Any errors in the identify function are logged and fallback to anonymous tracking
• Additional fields beyond userId are included in analytics for segmentation

Configuration Options

Option	Type	Default	Description
endpoint	string	"https://api.agnost.ai"	API endpoint URL
disableInput	boolean	false	Disable tracking of input arguments
disableOutput	boolean	false	Disable tracking of output results
disableLogs	boolean	false	Completely disable all SDK logs
identify	IdentifyFunction	undefined	Function to identify users from request context

Performance Monitoring with Checkpoints

The TypeScript SDK provides a powerful checkpoint() function for detailed latency breakup of tool calls. Checkpoints allow you to track specific points in your tool's execution, providing granular observability into where time is being spent.

Overview

When analyzing tool performance, knowing the total execution time is often not enough. The checkpoint() function lets you mark specific points in your execution flow to understand:

• Which operations are slow
• Where bottlenecks occur
• How time is distributed across different phases
• Performance impact of external API calls, database queries, or processing steps

All checkpoint data is automatically captured and visualized in the Agnost AI dashboard with interactive timeline charts.

Function Signature

import { checkpoint } from 'agnost';

checkpoint(name: string, metadata?: any): void

Parameters:

• name (string): A descriptive name for the checkpoint (e.g., "database_query_start", "api_call_complete")
• metadata (optional): Any additional context to attach to this checkpoint (e.g., row counts, response sizes, status codes)

Basic Usage

import { checkpoint } from 'agnost';
import { z } from 'zod';

// Define your tool
server.tool(
  'get_user_data',
  'Fetches and processes user data from the database',
  {
    userId: z.string().describe('The user ID to fetch')
  },
  async ({ userId }) => {
    // Mark the start of input validation
    checkpoint('input_validation_start');

    if (!userId || userId.length === 0) {
      throw new Error('Invalid user ID');
    }

    checkpoint('input_validation_complete');

    // Mark the start of database query
    checkpoint('database_query_start');

    const userData = await db.query('SELECT * FROM users WHERE id = ?', [userId]);

    checkpoint('database_query_complete', {
      rowCount: userData.length
    });

    // Mark the start of data processing
    checkpoint('data_processing_start');

    const processed = await processUserData(userData);

    checkpoint('data_processing_complete', {
      recordsProcessed: processed.length
    });

    return {
      content: [
        {
          type: 'text',
          text: JSON.stringify(processed)
        }
      ]
    };
  }
);

Advanced Example: API Call Monitoring

import { checkpoint } from 'agnost';

server.tool(
  'fetch_weather',
  'Fetches weather data from external API',
  {
    city: z.string()
  },
  async ({ city }) => {
    // Track input normalization
    checkpoint('input_normalization_start');
    const normalizedCity = city.trim().toLowerCase();
    checkpoint('input_normalization_complete');

    // Track cache lookup
    checkpoint('cache_lookup_start');
    const cached = await cache.get(`weather:${normalizedCity}`);
    checkpoint('cache_lookup_complete', { cacheHit: !!cached });

    if (cached) {
      checkpoint('returning_cached_data');
      return cached;
    }

    // Track external API call
    checkpoint('api_call_start');
    const response = await fetch(`https://api.weather.com/v1/${normalizedCity}`);
    checkpoint('api_call_complete', {
      statusCode: response.status,
      responseSize: response.headers.get('content-length')
    });

    // Track response parsing
    checkpoint('response_parsing_start');
    const data = await response.json();
    checkpoint('response_parsing_complete');

    // Track cache update
    checkpoint('cache_update_start');
    await cache.set(`weather:${normalizedCity}`, data, 3600);
    checkpoint('cache_update_complete');

    return {
      content: [
        {
          type: 'text',
          text: JSON.stringify(data)
        }
      ]
    };
  }
);

How Checkpoints Work

• Automatic Context Tracking: When a tool is called, the SDK automatically creates an execution context using AsyncLocalStorage
• Relative Timestamps: Each checkpoint records the time elapsed since the tool execution started (in milliseconds)
• Metadata Capture: Optional metadata is stored with each checkpoint for additional context
• Safe Operation: Checkpoints called outside of tool execution are safely ignored (no errors thrown)
• Zero Performance Impact: Checkpoints are optimized for minimal overhead and won't affect your tool's performance

Checkpoint Data Structure

Each checkpoint is recorded with the following structure:

interface Checkpoint {
  name: string;           // The checkpoint name
  timestamp: number;      // Milliseconds since execution start
  metadata?: any;         // Optional metadata object
}

Dashboard Visualization

Checkpoints are automatically visualized in the Agnost AI dashboard with:

• Timeline Bar Chart: Visual representation of time spent between checkpoints
• Detailed Breakdown: List of all checkpoints with:
  • Absolute timestamp (ms from start)
  • Duration since previous checkpoint
  • Percentage of total latency
  • Metadata display
• Remaining Time Analysis: Shows overhead/time not covered by explicit checkpoints

Example timeline visualization:

[0ms--------50ms][50ms---------200ms][200ms----250ms]
  Input Valid    DB Query           Processing

Best Practices

• Use Descriptive Names: Make checkpoint names clear and specific

  // Good
  checkpoint('database_query_complete');
  checkpoint('external_api_call_start');
  
  // Avoid
  checkpoint('step1');
  checkpoint('done');
  

• Track Start and End: For operations you want to measure, add both start and end checkpoints

  checkpoint('operation_start');
  await expensiveOperation();
  checkpoint('operation_complete');
  

• Add Useful Metadata: Include context that helps debug performance issues

  checkpoint('query_complete', {
    rowCount: results.length,
    queryTime: Date.now() - startTime,
    cacheHit: false
  });
  

• Focus on Expensive Operations: Add checkpoints around:

  • Database queries
  • External API calls
  • File I/O operations
  • Heavy computation
  • Network requests

• Don't Over-checkpoint: Too many checkpoints can make analysis harder. Focus on meaningful boundaries

  // Good: Major operation boundaries
  checkpoint('fetch_data_start');
  checkpoint('fetch_data_complete');
  checkpoint('process_data_complete');
  
  // Avoid: Too granular
  checkpoint('variable_declared');
  checkpoint('loop_iteration_1');
  checkpoint('loop_iteration_2');
  

Common Patterns

Pattern 1: Database Operations

checkpoint('db_connection_start');
const connection = await pool.getConnection();
checkpoint('db_connection_acquired');

checkpoint('db_query_start');
const results = await connection.query(sql);
checkpoint('db_query_complete', { rowCount: results.length });

Pattern 2: Multi-Step Processing Pipeline

checkpoint('fetch_raw_data');
const raw = await fetchData();

checkpoint('transform_data');
const transformed = transform(raw);

checkpoint('validate_data');
const validated = validate(transformed);

checkpoint('store_data');
await store(validated);
checkpoint('pipeline_complete');

Pattern 3: Parallel Operations

checkpoint('parallel_operations_start');

const [result1, result2, result3] = await Promise.all([
  operation1(),
  operation2(),
  operation3()
]);

checkpoint('parallel_operations_complete', {
  operation1Time: result1.duration,
  operation2Time: result2.duration,
  operation3Time: result3.duration
});

Troubleshooting

Checkpoints not appearing in dashboard:

• Ensure you're calling checkpoint() inside a tracked tool handler
• Verify analytics tracking is enabled with trackMCP()
• Check that your organization ID is correct

Timestamps seem incorrect:

• Timestamps are relative to tool execution start (not absolute time)
• Ensure you're not calling checkpoint() outside of tool execution context

Performance concerns:

• Checkpoints have minimal overhead (< 1ms per checkpoint)
• They use object pooling and efficient timestamp recording
• Safe to use even in high-frequency tools