Fast & Accurate: High-precision timing with statistical analysis
Multiple Output Formats: Human-readable, JSON, and CSV reports (at the same time!!)
Historical Tracking: Store and compare benchmark results over time
Code Profiling & Analysis: Identify hot code paths that need benchmarking using V8's built-in profiler
Performance Budgets: Enforce performance standards and prevent regressions
CLI & API: Command-line interface and programmatic API
TypeScript Support: Full type safety

In summary, modestbench wraps tinybench and enhances it with a bunch of crap so you don't have to think.

Quick Start

Installation

The usual suspects:

npm install --save-dev modestbench

Optional: Initialize a Project

The modestbench CLI provides a init command. This command:

Generates a configuration file in a format of your choosing
Creates an example benchmark file
Appends .modestbench/ to .gitignore to exclude historical data from version control

# Initialize with examples and configuration
modestbench init

# Or specify project type and config format
modestbench init advanced --config-type typescript

Project Types:

basic - Simple setup for small projects (100 iterations, human reporter)
advanced - Feature-rich with multiple reporters and structured output (1000 iterations, warmup, human + JSON reporters)
library - Optimized for library performance testing (5000 iterations, high warmup, human + JSON reporters, organized suite structure)

My First Benchmark

PRO TIP: The convention for modestbench benchmark files is to use the .bench.js or .bench.ts extension.

modestbench supports two formats for defining benchmarks:

Simplified Format (Recommended for Simple Cases)

For quick benchmarks with just a few tasks, you can use the simplified format:

// benchmarks/example.bench.js
export default {
  'Array.push()': () => {
    const arr = [];
    for (let i = 0; i < 1000; i++) {
      arr.push(i);
    }
    return arr;
  },

  'Array spread': () => {
    let arr = [];
    for (let i = 0; i < 1000; i++) {
      arr = [...arr, i];
    }
    return arr;
  },
};

Suite-Based Format (For Complex Projects)

When you need to organize benchmarks into groups with setup/teardown hooks:

// benchmarks/example.bench.js
export default {
  suites: {
    'Array Operations': {
      benchmarks: {
        'Array.push()': () => {
          const arr = [];
          for (let i = 0; i < 1000; i++) {
            arr.push(i);
          }
          return arr;
        },

        'Array spread': () => {
          let arr = [];
          for (let i = 0; i < 1000; i++) {
            arr = [...arr, i];
          }
          return arr;
        },
      },
    },
  },
};

When to use each format:

Simplified format: Quick benchmarks, single file with related tasks, no setup/teardown needed

Suite format: Complex projects, multiple groups of benchmarks, need setup/teardown hooks, or want explicit organization

Running Your First Benchmarks

# Run all benchmarks
modestbench

# Run with specific options
modestbench --iterations 5000 --reporter human --reporter json

View Results

Example output showing colorful terminal display with benchmark results

Getting Started

Jump to:

Quick Start - Basic concepts and your first benchmark
Configuration - Project and runtime configuration options
Advanced Features - Multiple suites, async operations, and tagging
Integration Examples - CI/CD integration and performance monitoring
Programmatic API - Using modestbench programmatically

See the examples directory for additional guides and sample code.

CLI Commands

Run Benchmarks

Run benchmarks with sensible defaults:

# Run benchmarks in current directory and bench/ (top-level only)
modestbench

# Run all benchmarks in a directory (searches recursively)
modestbench benchmarks/

# Run benchmarks from multiple directories
modestbench src/perf/ tests/benchmarks/

# Run specific files
modestbench benchmarks/critical.bench.js

# Mix files, directories, and glob patterns
modestbench file.bench.js benchmarks/ "tests/**/*.bench.ts"

# With options
modestbench \
  --config ./config.json \
  --iterations 2000 \
  --reporter human \
  --reporter json \
  --reporter csv \
  --output ./results \
  --tag performance \
  --tag algorithm \
  --concurrent

Supported file extensions:

JavaScript: .js, .mjs, .cjs
TypeScript: .ts, .mts, .cts

Controlling Benchmark Limits

The --limit-by flag controls whether benchmarks are limited by time, iteration count, or both:

# Limit by iteration count (fast, predictable sample size)
modestbench --iterations 100

# Limit by time budget (ensures consistent time investment)
modestbench --time 5000

# Limit by whichever comes first (safety bounds)
modestbench --iterations 1000 --time 10000

# Explicit control (overrides smart defaults)
modestbench --iterations 500 --time 5000 --limit-by time

# Require both thresholds (rare, for statistical rigor)
modestbench --iterations 100 --time 2000 --limit-by all

Smart Defaults:

Only --iterations provided → limits by iteration count (fast)
Only --time provided → limits by time budget
Both provided → stops at whichever comes first (any mode)
Neither provided → uses default iterations (100) with iterations mode

Modes:

iterations: Stop after N samples (time budget set to 1ms)
time: Run for T milliseconds (collect as many samples as possible)
any: Stop when either threshold is reached (defaults to iterations behavior for fast completion)
all: Require both time AND iterations thresholds (tinybench default behavior)

Filtering by Tags

Run specific subsets of benchmarks using tags:

# Run only benchmarks tagged with 'fast'
modestbench --tag fast

# Run benchmarks with multiple tags (OR logic - matches ANY)
modestbench --tag string --tag array --tag algorithm

# Exclude specific benchmarks
modestbench --exclude-tag slow --exclude-tag experimental

# Combine: run fast benchmarks except experimental ones
modestbench --tag fast --exclude-tag experimental

Key Features:

Tags cascade from file → suite → task levels
--tag uses OR logic when specified multiple times (matches ANY specified tag)
--exclude-tag takes precedence over --tag
Suite setup/teardown only runs if at least one task matches

See Tagging and Filtering for detailed examples.

Output Options

Control where and how benchmark results are saved:

# Write to a directory (creates results.json, results.csv, etc.)
modestbench --reporter json --reporter csv --output ./results

# Custom filename for single reporter
modestbench --reporter json --output-file my-benchmarks.json

# Custom filename in specific directory
modestbench --reporter json --output ./results --output-file benchmarks-2024.json

# Custom filename with absolute path
modestbench --reporter json --output-file /tmp/my-benchmarks.json

# With subdirectories
modestbench --reporter csv --output ./results --output-file reports/performance.csv

# Short flag alias (using -r for --reporter)
modestbench -r json --of custom.json

Key Options:

--output <dir>, -o <dir> - Directory to write output files (default: stdout)
--output-file <filename>, --of <filename> - Custom filename for output
- Works with absolute or relative paths
- Requires exactly one reporter (e.g., --reporter json)
- When used with --output, the filename is relative to that directory
- When used alone, the path is relative to current working directory

Limitations:

--output-file only works with a single reporter
For multiple reporters, use --output <dir> (defaults to results.json, results.csv, etc.)

History Management

modestbench automatically tracks benchmark results over time in a local .modestbench/ directory. This history enables you to:

Track performance trends - See how your code's performance changes across commits
Detect regressions - Compare current results against previous runs to catch slowdowns
Analyze improvements - Validate that optimizations actually improve performance
Document progress - Export historical data for reports and analysis

# List recent runs
modestbench history list

# Show detailed results
modestbench history show <run-id>

# Compare two runs
modestbench history compare <run-id-1> <run-id-2>

# Export historical data
modestbench history export --format csv --output results.csv

# Clean old data
modestbench history clean --older-than 30d

Performance Budgets

modestbench supports performance budgets to prevent regressions and enforce performance standards in CI/CD.

Configuring Budgets

Define budgets in your modestbench.config.json:

{
  "budgetMode": "fail",
  "budgets": {
    "benchmarks/critical.bench.js/default/parseConfig": {
      "absolute": {
        "maxTime": "10ms",
        "minOpsPerSec": 100000
      }
    }
  }
}

Budget Types:

Absolute Budgets: Fixed thresholds
- maxTime - Maximum mean execution time (e.g., "10ms", "500us")
- minOpsPerSec - Minimum operations per second
- maxP99 - Maximum 99th percentile latency
Relative Budgets: Comparison against baseline
- maxRegression - Maximum performance degradation (e.g., "10%", 0.1)

Budget Modes:

fail (default) - Exit with error code if budgets fail
warn - Show warnings but don't fail
report - Include in output without failing

Managing Baselines

# Save current run as a baseline
modestbench baseline set production-v1.0

# List all baselines
modestbench baseline list

# Compare against a baseline
modestbench run --baseline production-v1.0

# Analyze history and suggest budgets
modestbench baseline analyze

Profile Code Execution

Identify hot code paths that are good candidates for benchmarking:

# Profile a command
modestbench analyze "npm test"

# Profile a specific script
modestbench analyze "node ./src/server.js"

# Analyze existing profile
modestbench analyze --input isolate-0x123-v8.log

# Filter and customize output
modestbench analyze "npm test" \
  --filter-file "src/**" \
  --min-percent 2.0 \
  --top 50 \
  --group-by-file

Profile Options:

[command] - Command to profile (e.g., npm test, node script.js)
--input, -i - Path to existing V8 profile log file
--filter-file - Filter functions by file glob pattern
--min-percent - Minimum execution percentage to show (default: 1.0)
--top, -n - Number of top functions to show (default: 25)
--group-by-file - Group results by source file
--color - Enable/disable color output

How It Works:

The profile command uses Node.js's built-in V8 profiler to identify functions that consume the most execution time. It automatically filters out Node.js internals and node_modules to focus on your code.

Functions that appear at the top of the profile report are good candidates for benchmarking, as optimizing them will have the most impact on overall performance.

Configuration

Project Configuration

Create modestbench.config.json:

{
  "bail": false, // Stop execution on first failure
  "exclude": ["node_modules/**"], // Patterns to exclude from discovery
  "excludeTags": ["slow", "experimental"], // Tags to exclude from execution
  "iterations": 1000, // Number of samples per benchmark
  "limitBy": "iterations", // Limit mode: 'iterations', 'time', 'any', 'all'
  "outputDir": "./benchmark-results", // Directory for results and reports
  "pattern": "benchmarks/**/*.bench.{js,ts}", // Glob pattern to discover benchmark files
  "quiet": false, // Minimal output mode
  "reporters": ["human", "json"], // Output reporters to use
  "tags": ["fast", "critical"], // Tags to include (if empty, all benchmarks run)
  "time": 5000, // Time budget in ms per benchmark
  "timeout": 30000, // Task timeout in ms
  "verbose": false, // Detailed output with debugging info
  "warmup": 50, // Warmup iterations before measurement
}

Configuration Options:

pattern - Glob pattern(s) to discover benchmark files (can be string or array)
exclude - Glob patterns for files/directories to exclude from discovery
excludeTags - Array of tags to exclude from execution; benchmarks with ANY of these tags will be skipped (default: [])
iterations - Number of samples to collect per benchmark task (default: 100)
time - Time budget in milliseconds per benchmark task (default: 1000)
limitBy - How to limit benchmarks: "iterations" (sample count), "time" (time budget), "any" (whichever comes first), or "all" (both thresholds required)
warmup - Number of warmup iterations before measurement begins (default: 0)
timeout - Maximum time in milliseconds for a single task before timing out (default: 30000)
bail - Stop execution on first benchmark failure (default: false)
reporters - Array of reporter names to use for output (available: "human", "json", "csv")
outputDir - Directory path for saving benchmark results and reports
quiet - Minimal output mode, suppresses non-essential messages (default: false)
tags - Array of tags to include; if non-empty, only benchmarks with ANY of these tags will run (default: [])
verbose - Detailed output mode with additional debugging information (default: false)

Note: Smart defaults apply for limitBy based on which options you provide. See Controlling Benchmark Limits for details.

Configuration File Support

modestbench supports multiple configuration file formats, powered by cosmiconfig:

JSON: modestbench.config.json, .modestbenchrc.json, .modestbenchrc
YAML: modestbench.config.yaml, modestbench.config.yml, .modestbenchrc.yaml, .modestbenchrc.yml
JavaScript: modestbench.config.js, modestbench.config.mjs, .modestbenchrc.js, .modestbenchrc.mjs
TypeScript: modestbench.config.ts
package.json: Use a "modestbench" field

Generate a configuration file using:

modestbench init --config-type json   # JSON format
modestbench init --config-type yaml   # YAML format
modestbench init --config-type js     # JavaScript format
modestbench init --config-type ts     # TypeScript format

Configuration Discovery: modestbench automatically searches for configuration files in the current directory and parent directories, following standard conventions.

Output Formats

Human-Readable (Default)

Real-time progress bars with color-coded results and performance summaries.

JSON Output

Structured data perfect for programmatic analysis and integration:

{
  "results": [
    {
      "file": "example.bench.js",
      "suite": "Array Operations",
      "task": "Array.push()",
      "hz": 1234567.89,
      "stats": {
        "mean": 0.00081,
        "stdDev": 0.00002,
        "marginOfError": 2.45
      }
    }
  ],
  "run": {
    "id": "run-2025-10-07-001",
    "timestamp": "2025-10-07T10:30:00.000Z",
    "duration": 15420,
    "status": "completed"
  }
}

CSV Export

Tabular data for spreadsheet analysis and historical tracking.

Advanced Features

Multiple Suites

const state = {
  data: [],
  sortedData: [],
};

export default {
  suites: {
    Sorting: {
      setup() {
        state.data = generateTestData(1000);
      },
      benchmarks: {
        // Shorthand syntax for simple benchmarks
        'Quick Sort': () => quickSort(state.data),
        'Merge Sort': () => mergeSort(state.data),
      },
    },

    Searching: {
      setup() {
        state.sortedData = generateSortedData(10000);
      },
      benchmarks: {
        'Binary Search': () => binarySearch(state.sortedData, 5000),
        'Linear Search': () => linearSearch(state.sortedData, 5000),
      },
    },
  },
};

Async Operations

export default {
  suites: {
    'Async Performance': {
      benchmarks: {
        // Shorthand syntax works with async functions too
        'Promise.resolve()': async () => {
          return await Promise.resolve('test');
        },

        // Full syntax when you need config, tags, or metadata
        'Fetch Simulation': {
          async fn() {
            const response = await simulateApiCall();
            return response.json();
          },
          config: {
            iterations: 100, // Fewer iterations for slow operations
          },
        },
      },
    },
  },
};

Tagging and Filtering

modestbench supports a powerful tagging system that lets you organize and selectively run benchmarks. Tags can be applied at three levels: file, suite, and task. Tags automatically cascade from parent to child, so tasks inherit tags from their suite and file.

Adding Tags

Tags can be added at any level:

export default {
  // File-level tags (inherited by all suites and tasks)
  tags: ['performance', 'core'],

  suites: {
    'String Operations': {
      // Suite-level tags (inherited by all tasks in this suite)
      tags: ['string', 'fast'],

      benchmarks: {
        // Task inherits: ['performance', 'core', 'string', 'fast', 'regex']
        'RegExp Test': {
          fn: () => /pattern/.test(str),
          tags: ['regex'], // Task-specific tags
        },

        // Task inherits: ['performance', 'core', 'string', 'fast']
        'String Includes': () => str.includes('pattern'),
      },
    },
  },
};

Filtering Benchmarks

Use --tag (or -t) to include only benchmarks with specific tags (OR logic - matches ANY tag):

# Run fast algorithms
modestbench run --tag fast

# Run benchmarks tagged with 'string' OR 'array'
modestbench run --tag string --tag array

# Using short aliases
modestbench run -t fast -t optimized

Use --exclude-tag (or -e) to skip benchmarks with specific tags:

# Exclude slow benchmarks
modestbench run --exclude-tag slow

# Exclude experimental and unstable benchmarks
modestbench run --exclude-tag experimental --exclude-tag unstable

Combine both to fine-tune your benchmark runs (exclusion takes precedence):

# Run fast benchmarks, but exclude experimental ones
modestbench run --tag fast --exclude-tag experimental

# Run algorithm benchmarks but skip slow reference implementations
modestbench run --tag algorithm --exclude-tag slow --exclude-tag reference

Tag Cascading Example

export default {
  tags: ['file-level'], // All tasks get this tag

  suites: {
    'Fast Suite': {
      tags: ['fast'], // Tasks get: ['file-level', 'fast']
      benchmarks: {
        'Task A': {
          fn: () => {},
          tags: ['math'], // This task has: ['file-level', 'fast', 'math']
        },
        'Task B': () => {}, // This task has: ['file-level', 'fast']
      },
    },
  },
};

Filtering Behavior:

--tag math → Runs only Task A (matches 'math')
--tag fast → Runs both Task A and Task B (both have 'fast')
--tag file-level → Runs both tasks (both inherit 'file-level')
--exclude-tag math → Runs only Task B (Task A excluded)

Suite Lifecycle with Filtering

Suite setup() and teardown() only run if at least one task in the suite matches the filter. This prevents unnecessary setup work for filtered-out suites.

Integration Examples

GitHub Actions

name: Performance Tests
on: [push, pull_request]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18

      - run: npm ci
      - run: npm run build

      - name: Run Benchmarks
        run: |
          modestbench \
            --reporter json \
            --reporter csv \
            --output ./results

      - name: Upload Results
        uses: actions/upload-artifact@v3
        with:
          name: benchmark-results
          path: ./results/

Performance Regression Detection

// scripts/check-regression.js
import { execSync } from 'child_process';
import { readFileSync } from 'fs';

// Run current benchmarks
execSync('modestbench --reporter json --output ./current');
const current = JSON.parse(readFileSync('./current/results.json'));

// Load baseline results
const baseline = JSON.parse(readFileSync('./baseline/results.json'));

// Check for significant regressions
for (const result of current.results) {
  const baselineResult = baseline.results.find(
    (r) => r.file === result.file && r.task === result.task,
  );

  if (baselineResult) {
    const regression = (baselineResult.hz - result.hz) / baselineResult.hz;
    if (regression > 0.1) {
      // 10% regression threshold
      console.error(
        `Performance regression detected in ${result.task}: ${(regression * 100).toFixed(1)}%`,
      );
      process.exit(1);
    }
  }
}

console.log('No performance regressions detected ✅');

Programmatic API

import { modestbench, HumanReporter } from 'modestbench';

// initialize the engine
const engine = modestbench();

engine.registerReporter('human', new HumanReporter());

// Execute benchmarks
const result = await engine.execute({
  pattern: '**/*.bench.js',
  iterations: 1000,
});

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone the repository
git clone https://github.com/boneskull/modestbench.git
cd modestbench

# Install dependencies
npm install

# Run tests
npm test

# Build the project
npm run build

# Run examples
npm run examples

Acknowledgments

Built on top of the small-but-mighty benchmarking library, tinybench
Interface inspired by good ol' Benchmark.js
Built with zshy for dual ESM/CJS modules
AccurateEngine statistical analysis inspired by the excellent work of bench-node

Resources

License

Keywords

FAQs

What is modestbench?

Is modestbench popular?

Is modestbench well maintained?

Package last updated on 30 Oct 2025

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

modestbench

Features