@copilotkit/pathfinder

pathfinder

The knowledge server for AI agents — index your docs, code, Notion pages, Slack threads, and Discord forums into searchable, agent-accessible knowledge via MCP. One config file, one command, works with any AI coding agent.

Quick Start

npx @copilotkit/pathfinder init
npx @copilotkit/pathfinder serve

Or with Docker:

docker pull ghcr.io/copilotkit/pathfinder
docker run -v ./pathfinder.yaml:/app/pathfinder.yaml \
  -v ./docs:/app/docs -p 3001:3001 \
  ghcr.io/copilotkit/pathfinder

For local (transformers.js) embeddings in Docker, use the ghcr.io/copilotkit/pathfinder:latest-local image — it ships @xenova/transformers preinstalled. The default image stays slim and fails loudly at startup if embedding.provider: local is set without the dependency.

Then connect your AI agent:

{
  "mcpServers": {
    "my-docs": { "url": "http://localhost:3001/mcp" }
  }
}

Try It — Pathfinder on Its Own Docs

This documentation is indexed by a live Pathfinder instance. Connect your agent to try it:

# Claude Code
claude mcp add pathfinder-docs --transport http https://mcp.pathfinder.copilotkit.dev/mcp

// Claude Desktop / Cursor / any MCP client
{
  "mcpServers": {
    "pathfinder-docs": {
      "url": "https://mcp.pathfinder.copilotkit.dev/mcp"
    }
  }
}

What It Does

Pathfinder indexes your GitHub repos — docs (Markdown, MDX, HTML) and source code — into a PostgreSQL vector database. Supports OpenAI, Ollama, and local transformers.js embeddings — no API key required for local providers. It serves configurable search and filesystem exploration tools via MCP, so AI agents can search your docs semantically and browse files with bash commands.

Tool Type	What It Does	Example
Search	Semantic search over indexed content	search-docs("how to authenticate")
Bash	Virtual filesystem with find, grep, cat, ls	explore-docs("cat /docs/quickstart.mdx")
Collect	Structured data collection from agents	submit-feedback(rating: "helpful")
Knowledge	Browse/search FAQ pairs from conversational sources	knowledge-base("how to deploy")

Features

• Semantic Search — pgvector RAG with configurable chunk sizes, overlap, and score thresholds
• Filesystem Exploration — QuickJS WASM sandbox with session state, qmd semantic grep, related files
• 8 Source Types — Markdown, code, raw-text, HTML, document (PDF/DOCX), Slack, Discord, Notion — with pluggable chunker registry
• Hybrid Search — Combine vector and keyword search with RRF scoring for better recall on technical terms
• Multiple Embedding Providers — OpenAI, Ollama (local HTTP), or transformers.js (zero external deps, CPU-only)
• Config-Driven — Everything in one pathfinder.yaml: sources, tools, embedding, indexing, webhooks
• Client Setup — Claude Desktop, Claude Code, Cursor, Codex, VS Code, any Streamable HTTP client
• Docker + Railway — Container image, docker-compose, Railway one-click
• Conversational Sources — Slack threads and Discord forums distilled into searchable Q&A pairs
• Auto-Generated Endpoints — /llms.txt, /llms-full.txt, /faq.txt, /.well-known/skills/default/skill.md
• Webhook Reindexing — GitHub push triggers incremental reindex
• Session Management — Global session cap, per-IP rate limiting, two-tier TTL (active vs unused sessions)
• Analytics — Query logging, top queries, empty results, latency metrics at /analytics

CLI

# Scaffold config
npx @copilotkit/pathfinder init

# Auto-generate config from an existing docs site
npx @copilotkit/pathfinder init --from <url>

# Start server (uses PGlite if no DATABASE_URL)
npx @copilotkit/pathfinder serve

# Validate config, env vars, and source connectivity
npx @copilotkit/pathfinder validate

# List every missing required env var in one pass (Atlas CLI)
# The `atlas` bin ships in the same package; --package selects it since the
# bin name differs from the package name.
npx --package @copilotkit/pathfinder atlas preflight

# Docker with Postgres
docker compose up

Required environment variables

Requirements are conditional — a var is only required when the config or runtime mode actually needs it. Run npx --package @copilotkit/pathfinder atlas preflight to report every missing one in a single pass before starting the server.

Fatal at boot when NODE_ENV=production (in development both are generated ephemerally with a warning):

Variable	Generate with	Purpose
MCP_JWT_SECRET	openssl rand -hex 32	Signs MCP session JWTs
PATHFINDER_CONSENT_HMAC_KEY	openssl rand -hex 32	Signs the OAuth consent-screen nonce; comma-separate values to rotate keys

Source-gated — required only when a matching source is configured:

Source type	Required variables
slack	SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET, OPENAI_API_KEY
discord	DISCORD_BOT_TOKEN, DISCORD_PUBLIC_KEY (+ OPENAI_API_KEY for text channels)
notion	NOTION_TOKEN

DATABASE_URL and OPENAI_API_KEY are required whenever search/knowledge/ collect tools or semantic bash grep are configured.

Switching from Mintlify?

Step-by-step migration guide: Migrate from Mintlify

Telemetry

Self-hosted Pathfinder sends nothing externally. No phone-home, no analytics. The telemetry code path is gated on a CopilotKit-internal env var that isn't set in any image you pull or any package you install — running your own copy is opt-out by default, with no flag to flip.

The hosted instance at mcp.pathfinder.copilotkit.dev records one of two events, depending on the protocol the MCP client uses. The server sends these events; the client sends nothing extra.

For session-based connections, the server sends one event named pathfinder.session.created per MCP client connection — when a fresh client (claude.ai, Cursor, a custom MCP client) opens a session against the hosted server. It contains:

• the client's IP address
• the User-Agent string
• the MCP transport in use (sse or streamable_http)
• the first 8 characters of the session ID (for log correlation)
• whether the client presented an OAuth bearer token

Connections that use the 2026-07-28 (stateless) protocol have no session, so for them the server sends a separate event named pathfinder.client.seen. It is sent only when PATHFINDER_MODERN_PROTOCOL is enabled on the server (it is off by default), and, on a best-effort basis, once per IP address and User-Agent per 24 hours per server process (the server tracks up to 10,000 recently seen clients, so an evicted client can be sent again sooner). It contains:

• the client's IP address
• the User-Agent string, cut to its first 256 bytes (Node reads header values as latin1, one character per byte, so this is 256 characters for an ASCII User-Agent, and a cut can split a multi-byte UTF-8 character)
• the MCP transport (streamable_http)
• the protocol era (modern)
• whether the client presented an OAuth bearer token

It has no session ID prefix.

What neither event contains: search queries, knowledge tool inputs, response content, full session IDs, or JWT subjects — anything inside an MCP request stays on the hosted server. There is no per-tool-call event of any kind.

Both events are sent to a CopilotKit-controlled endpoint and forwarded to a small set of third-party analytics providers, which use the IP for company-level attribution — i.e., figuring out which organizations are evaluating the hosted instance. There's no individual-user identification; the IP is the only personal data point in the payload, and it's processed by those providers per their published terms.

If you'd rather not have any of this happen, run your own copy — Quick Start above gets you there in two commands. Same code, same features, no telemetry env vars set.

Documentation

https://pathfinder.copilotkit.dev

License

Pathfinder is source-available under the Elastic License 2.0 (ELv2) with an Additional Use Grant.

You can: use it, modify it, self-host it, host it for your project's docs, run it for your company, contribute to it — all free.

One restriction: you can't sell Pathfinder as a standalone product or service. That's it.

See LICENSING.md for plain-English details.