json-batch-processor-mcp
Advanced tools
MCP Server for batch processing large JSON arrays in manageable batches with progress tracking
A Model Context Protocol (MCP) server that enables AI IDEs to process large JSON arrays in manageable batches. This tool solves the context window limitation problem by breaking down large datasets into smaller chunks that can be processed incrementally.
npm install -g json-batch-processor-mcp
git clone <repository-url>
cd json-batch-processor-mcp
npm install
npm run build
npm link
Add the following configuration to your MCP settings file:
.kiro/settings/mcp.json (project-specific)~/.kiro/settings/mcp.json (global, all projects)If you cloned the repository or installed from source:
{
"mcpServers": {
"json-batch-processor": {
"command": "node",
"args": ["/absolute/path/to/json-batch-processor-mcp/dist/index.js"],
"disabled": false,
"autoApprove": []
}
}
}
Replace /absolute/path/to/json-batch-processor-mcp with the actual path to your installation.
If you installed globally via npm install -g:
{
"mcpServers": {
"json-batch-processor": {
"command": "json-batch-processor",
"args": [],
"disabled": false,
"autoApprove": []
}
}
}
true to disable the server without removing the configuration["bind_json_file", "read_batch"])After updating the configuration:
The server stores all data in:
~/.kiro/mcp-data/json-batch-processor/
Each binding creates a subdirectory with:
binding.json - Binding metadatatasks.json - Task listprogress.json - Progress trackingresults/ - Processed batch resultsBind a JSON file for batch processing.
Parameters:
filePath (string, required): Absolute path to the JSON fileReturns:
{
"bindingId": "uuid-string",
"structure": {
"type": "object",
"arrayPaths": ["$.data", "$.items"],
"totalElements": 1000
}
}
Generate a list of batch processing tasks.
Parameters:
bindingId (string, required): The binding ID from bind_json_filebatchSize (number, required): Number of elements per batchfieldPath (string, optional): JSONPath to the array (e.g., "$.data.items")Returns:
{
"bindingId": "uuid-string",
"totalTasks": 20,
"batchSize": 50,
"fieldPath": "$.data",
"tasks": [
{
"id": "uuid-batch-0",
"batchIndex": 0,
"startIndex": 0,
"endIndex": 49,
"status": "pending"
}
]
}
Read data for a specific batch.
Parameters:
bindingId (string, required): The binding IDtaskId (string, required): The task ID from the task listReturns:
{
"taskId": "uuid-batch-0",
"batchIndex": 0,
"data": [...],
"startIndex": 0,
"endIndex": 49,
"totalElements": 1000
}
Update the status of a task and optionally save processed results.
Parameters:
bindingId (string, required): The binding IDtaskId (string, required): The task IDstatus (string, required): Either "pending" or "completed"result (array, optional): The processed batch dataReturns:
{
"success": true,
"taskId": "uuid-batch-0",
"status": "completed"
}
Get the current processing progress.
Parameters:
bindingId (string, required): The binding IDReturns:
{
"bindingId": "uuid-string",
"totalTasks": 20,
"completedTasks": 5,
"pendingTasks": 15,
"percentage": 25,
"tasks": [
{
"taskId": "uuid-batch-0",
"status": "completed",
"completedAt": "2025-11-08T10:05:00Z",
"hasResult": true
}
]
}
Merge all processed batches into a final JSON file.
Parameters:
bindingId (string, required): The binding IDoutputPath (string, required): Path for the output JSON fileReturns:
{
"success": true,
"outputPath": "/path/to/output.json",
"totalElements": 1000,
"message": "Successfully merged 20 batches"
}
Here's a minimal example to get started:
// 1. Bind a JSON file
const binding = await callTool("bind_json_file", {
filePath: "/path/to/data.json"
});
// 2. Generate tasks (50 items per batch)
const tasks = await callTool("generate_task_list", {
bindingId: binding.bindingId,
batchSize: 50,
fieldPath: "$.items" // Optional: specify array path
});
// 3. Process each batch
for (const task of tasks.tasks) {
// Read batch
const batch = await callTool("read_batch", {
bindingId: binding.bindingId,
taskId: task.id
});
// Process data (your custom logic)
const processed = batch.data.map(item => ({
...item,
processed: true
}));
// Save results
await callTool("update_task_status", {
bindingId: binding.bindingId,
taskId: task.id,
status: "completed",
result: processed
});
}
// 4. Merge all results
const result = await callTool("merge_results", {
bindingId: binding.bindingId,
outputPath: "/path/to/output.json"
});
For a complete walkthrough with detailed examples, see EXAMPLE.md.
The server returns structured error responses:
{
"error": {
"code": "FileNotFoundError",
"message": "JSON file not found at path: /path/to/file.json",
"details": {}
}
}
Common error codes:
FileNotFoundError: JSON file doesn't existInvalidJSONError: Invalid JSON formatBindingNotFoundError: Binding ID not foundTaskNotFoundError: Task ID not foundInvalidBatchIndexError: Batch index out of rangeIncompleteTasksError: Not all tasks completedInvalidFieldPathError: Invalid JSONPath or path doesn't point to an arrayStorageError: File system operation failedThe server consists of six core components:
Data is stored in ~/.kiro/mcp-data/json-batch-processor/{bindingId}/:
binding.json - File binding metadatatasks.json - Complete task listprogress.json - Current progress stateresults/ - Individual batch resultsnpm run build
npm run dev
npm run build:clean
json-batch-processor-mcp/
├── src/
│ ├── index.ts # MCP server entry point
│ ├── managers/ # Core business logic
│ │ ├── BindingManager.ts
│ │ ├── TaskManager.ts
│ │ ├── BatchReader.ts
│ │ ├── ProgressTracker.ts
│ │ └── ResultMerger.ts
│ ├── types/ # TypeScript type definitions
│ │ └── index.ts
│ └── utils/ # Utility functions
│ ├── storage.ts
│ └── errors.ts
├── examples/ # Sample JSON files
├── dist/ # Compiled JavaScript
└── package.json
MIT
Contributions are welcome! Please open an issue or submit a pull request.
Fixed a critical bug where binding information was not persisted to disk, causing read_batch and subsequent operations to fail. The server now correctly saves binding.json files, ensuring reliable operation across server restarts.
What changed:
read_batch operations work reliably even after server restartMigration: If you have existing bindings from before this fix, please re-bind your JSON files using bind_json_file.
Problem: MCP server doesn't appear in your IDE
Solutions:
"disabled": false)Problem: FileNotFoundError when binding
Solutions:
ls -la /path/to/file.jsonProblem: InvalidJSONError when binding
Solutions:
cat file.json | jq .Problem: BindingNotFoundError on subsequent operations
Solutions:
~/.kiro/mcp-data/json-batch-processor/Problem: IncompleteTasksError when merging
Solutions:
get_progress to see which tasks are pendingQ: Can I process multiple JSON files simultaneously?
A: Yes! Each binding has a unique ID, so you can bind multiple files and process them in parallel.
Q: What happens if my process crashes?
A: All progress is saved to disk. Simply call get_progress to see where you left off and continue.
Q: Can I change the batch size after generating tasks?
A: No, you need to unbind and create a new binding with a different batch size.
Q: Does this work with nested arrays?
A: Yes! Use JSONPath syntax to specify the exact array path (e.g., $.data.users[*].orders).
Q: Can I process arrays in parallel?
A: Yes, tasks are independent. You can process multiple batches simultaneously if your logic allows.
Q: What if my JSON has multiple arrays?
A: Specify the exact array using the fieldPath parameter. If omitted, the first array is used.
Q: Can I modify the original file?
A: No, the original file is never modified. Results are always written to a new output file.
Q: How do I clean up old bindings?
A: Delete the binding directory: rm -rf ~/.kiro/mcp-data/json-batch-processor/{bindingId}
For issues and questions, please open an issue on the GitHub repository.
FAQs
MCP Server for batch processing large JSON arrays in manageable batches with progress tracking
The npm package json-batch-processor-mcp receives a total of 2 weekly downloads. As such, json-batch-processor-mcp popularity was classified as not popular.
We found that json-batch-processor-mcp demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Multiple high-impact npm maintainers confirm they have been targeted in the same social engineering campaign that compromised Axios.
Security News
Axios compromise traced to social engineering, showing how attacks on maintainers can bypass controls and expose the broader software supply chain.
Security News
Node.js has paused its bug bounty program after funding ended, removing payouts for vulnerability reports but keeping its security process unchanged.