mcpanvil

MCPAnvil - Where AI Agents Forge MCP Connections

MCPAnvil is a comprehensive directory of Model Context Protocol (MCP) servers, optimized for AI agents and developers. Live at mcpanvil.com.

Features

• 1,000+ MCPs Indexed: Comprehensive coverage from multiple sources
• 27 Categories: Smart categorization with only 13% uncategorized
• Agent-Optimized: JSON API designed for LLM consumption
• Queue-Based Crawler: Redis queue architecture prevents corruption
• Static Generation: Lightning fast, CDN-hosted, no backend
• Daily Updates: Automated crawling keeps data fresh

Architecture

GitHub Sources → Redis Queue → Workers → SQLite → Static Site → CloudFront CDN

Quick Start

Prerequisites

• Node.js 18+
• Redis (Docker preferred)
• GitHub token (for API rate limits)

Setup

• Install dependencies:

npm install

• Configure environment:

cp .env.example .env
# Add your GitHub token

• Start Redis:

docker-compose up -d

• Initialize database:

sqlite3 database/mcpanvil.db < database/schema.sql
node database/init-sources.js

• Run the crawler:

./scripts/start-crawler.sh
# Monitor with: node scripts/monitor-queues.js

• Build static site:

npm run build

• Deploy:

npm run deploy aws mcpanvil-site

API Endpoints

AI agents can access:

• /api/mcps.json - Complete MCP database
• /api/categories.json - Category statistics
• /api/by-tool/{tool}.json - MCPs by tool (claudeDesktop, vscode, etc.)

Example Usage

// Fetch all MCPs
fetch('https://mcpanvil.com/api/mcps.json')
  .then(r => r.json())
  .then(mcps => console.log(`Found ${mcps.length} MCPs`));

// Get categories
fetch('https://mcpanvil.com/api/categories.json')
  .then(r => r.json())
  .then(cats => console.log('Categories:', cats));

Data Schema

{
  "id": "github-owner-repo",
  "name": "mcp-name",
  "description": "What this MCP does",
  "repository": "https://github.com/owner/repo",
  "stars": 123,
  "categories": ["ai", "dev-tools"],
  "installation": {
    "claudeDesktop": { /* config */ },
    "vscode": { /* config */ },
    "npx": "npx -y mcp-name"
  },
  "last_crawled": "2025-07-25T19:09:23Z"
}

Project Structure

mcp-site/
├── crawler/           # Queue-based crawler system
│   ├── producer.js   # Discovers MCPs from sources
│   ├── worker.js     # Validates and fetches data
│   └── consumer.js   # Writes to database
├── database/         # SQLite database
│   ├── schema.sql    # Database structure
│   └── db.js        # Database abstraction
├── scripts/         # Build and deploy scripts
├── public/          # Static site assets
└── docs/           # Detailed documentation

Documentation

• Architecture - System design and data flow
• Crawler Guide - How the crawler works
• Categories - Category system documentation
• Deployment - AWS deployment guide

Key Features

Redis Queue Architecture

Prevents database corruption with single-writer pattern:

• Producer queues GitHub URLs
• Workers fetch in parallel
• Consumer writes sequentially

Smart Categorization

27 categories detected via keywords:

• Technical: ai, dev-tools, api, database, cloud
• Domain: finance, ecommerce, education, legal
• Media: audio, video, gaming, entertainment
• Only 13% remain uncategorized

Incremental Updates

• Tracks last_modified timestamps
• Preserves existing data
• Efficient daily refreshes

Contributing

• Add new sources to database/sources table
• Improve category keywords in crawler/consumer.js
• Enhance MCP detection logic in crawler/worker.js

Technical Stack

• Backend: Node.js, SQLite, Redis, Bull
• Frontend: Vanilla JS, No framework
• Infrastructure: AWS S3 + CloudFront
• CI/CD: Local cron jobs (for now)

License

MIT