fiftyone-mcp-server

FiftyOne MCP Server

 

Control FiftyOne datasets through AI assistants using the Model Context Protocol

Documentation · FiftyOne Skills · FiftyOne Plugins · Discord

What is the FiftyOne MCP Server?

Enable Agents to explore datasets, execute operators, and control the FiftyOne App through natural language. This server exposes 45+ MCP tools across data operations, App UI control, and the full operator/plugin ecosystem.

"List all my datasets"
"Load quickstart dataset and show summary"
"Find similar images in my dataset"

The server starts with 50 built-in operators. Install plugins to expand functionality - the AI can discover and install plugins automatically when needed (brain, zoo, annotation, evaluation, and more).

Available Tools

Category	Tools	Description
📊 Dataset Management	3	List, load, and summarize datasets
🎯 App Operations	29	Control the App UI (views, panels, selection, ...)
⚡ Operator System	3	Discover and execute any FiftyOne operator
🔄 Pipelines	2	Run pipelines and manage delegated operations
🔌 Plugin Management	5	Discover, install, and manage plugins
🖥️ Session	1	Launch the FiftyOne App server
📈 Aggregations	8	Count, distinct, bounds, mean, histogram, ...
🧬 Samples	5	Add, tag, untag, and set values on samples
🗂️ Schema	2	Inspect and modify dataset field schemas
🎨 App Config	6	Color scheme, sidebar groups, active fields

Tool modes

45+ tools organized by runtime mode:

• SDK: Data operations that work everywhere (datasets, aggregations, schema, samples, operators, plugins). No App connection needed.
• APP: Controls the FiftyOne App UI in real time (set_view, open_panel, notify, select_samples, reload, and 25+ more). Requires a connected browser via ctx.ops.
• SESSION: Bootstrap tools for starting a local App server (launch_app). Used from terminal environments.

Choosing your tools

Which tools are available depends on how you integrate the server:

Integration	Modes	Use case
FiftyOne App plugin	app + sdk	Agent panel inside the App (full UI control + data operations)
Terminal / CLI	session + sdk	Headless agent (launch the App, query data, execute operators)

Tool risk levels

Every tool is tagged with a risk level that your agent can use for auto-approval decisions:

• LOW Safe to auto-execute without prompting (read-only queries, UI state changes)
• OPERATOR Wraps a FiftyOne operator whose own severity should be checked before executing

Quick Start

Step 1: Install the MCP Server

pip install fiftyone-mcp-server

⚠️ Important: Make sure to use the same Python environment where you installed the MCP server when configuring your AI tool. If you installed it in a virtual environment or conda environment, you must activate that environment or specify the full path to the executable.

Step 2: Configure Your AI Tool

Claude Code (Recommended)

claude mcp add fiftyone -- fiftyone-mcp

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

Cursor

Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

VSCode

Add to .vscode/mcp.json:

{
  "servers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

ChatGPT Desktop

Edit ~/Library/Application Support/ChatGPT/config.json:

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

uvx (No Install Needed)

If you have uv installed:

{
  "mcpServers": {
    "fiftyone": {
      "command": "uvx",
      "args": ["fiftyone-mcp-server"]
    }
  }
}

This downloads and runs the latest version automatically.

Step 3: Use It

"List all my datasets"
"Load quickstart dataset and show summary"
"Open the map panel and show me the embeddings"
"Select samples with confidence above 0.9"
"What plugins are available? Install the brain plugin"
"Find near-duplicate images in my dataset"

Claude will automatically discover operators and execute the appropriate tools.

Contributing

We welcome contributions! Here's how to set up a local development environment:

• Clone the repository

  git clone https://github.com/voxel51/fiftyone-mcp-server.git
  cd fiftyone-mcp-server
  

• Install dependencies

  poetry install
  

• Run the server locally

  poetry run fiftyone-mcp
  

• Test your changes

  poetry run pytest
  poetry run black -l 79 src/
  npx @modelcontextprotocol/inspector poetry run fiftyone-mcp
  

• Submit a Pull Request

Resources

Resource	Description
FiftyOne Docs	Official documentation
FiftyOne Skills	Expert workflows for AI assistants
FiftyOne Plugins	Official plugin collection
Model Context Protocol	MCP specification
PyPI Package	MCP server on PyPI
Discord Community	Get help and share ideas

Community

Join the FiftyOne community to get help, share your ideas, and connect with other users:

• Discord: FiftyOne Community
• GitHub Issues: Report bugs or request features

Copyright 2017-2026, Voxel51, Inc. · Apache 2.0 License