mongo-pipeline-kit

Mongo Pipeline Kit

A feature-rich MongoDB pipeline builder kit for creating, reusing, and managing aggregation pipelines with ease. Now with enhanced JSON support, pipeline utilities, and advanced analysis capabilities.

Features

• 🏗️ Object-oriented pipeline building
• 🔄 Reusable pipeline components
• 🎯 Type-safe pipeline construction
• 📦 Pipeline composition and chaining
• 🔍 Built-in pipeline validation
• 📝 Comprehensive pipeline documentation
• 🧩 Modular and extensible design
• 📄 NEW: JSON export/import functionality
• 🛠️ NEW: Advanced pipeline utilities and analysis
• 📊 NEW: Pipeline statistics and complexity estimation
• 🔄 NEW: Pipeline comparison and manipulation tools

Installation

npm install mongo-pipeline-kit

Quick Start

import { PipelineBuilder } from "mongo-pipeline-kit";

// Create a new pipeline builder
const builder = new PipelineBuilder();

// Add stages to your pipeline
const pipeline = builder
  .match({ status: "active" })
  .group({
    _id: "$category",
    total: { $sum: 1 },
    items: { $push: "$$ROOT" },
  })
  .sort({ total: -1 })
  .limit(10)
  .build();

// Use the pipeline with MongoDB
const results = await collection.aggregate(pipeline).toArray();

JSON Export/Import

Export Pipeline to JSON

import { PipelineBuilder } from "mongo-pipeline-kit";

const builder = new PipelineBuilder()
  .match({ status: "active" })
  .group({ _id: "$category", count: { $sum: 1 } })
  .sort({ count: -1 })
  .limit(10);

// Get JSON string (compact)
const jsonString = builder.toJSON();
console.log(jsonString);
// Output: [{"$match":{"status":"active"}},{"$group":{"_id":"$category","count":{"$sum":1}}},{"$sort":{"count":-1}},{"$limit":10}]

// Get JSON string (pretty formatted)
const prettyJson = builder.toJSON(true);
console.log(prettyJson);
// Output: Formatted JSON with indentation

// Export with metadata
const exportData = builder.exportWithMetadata({
  description: "Active users by category",
  author: "John Doe",
  version: "1.0.0",
});

Import Pipeline from JSON

import { PipelineUtils } from "mongo-pipeline-kit";

const jsonString = `[
  {"$match": {"status": "active"}},
  {"$group": {"_id": "$category", "count": {"$sum": 1}}},
  {"$sort": {"count": -1}},
  {"$limit": 10}
]`;

// Parse JSON string to pipeline
const pipeline = PipelineUtils.fromJSON(jsonString);

// Use with PipelineBuilder
const builder = new PipelineBuilder();
pipeline.forEach((stage) => builder.addStage(stage));

Pipeline Utilities

Pipeline Analysis

import { PipelineUtils } from "mongo-pipeline-kit";

const pipeline = [
  { $match: { status: "active" } },
  { $group: { _id: "$category", count: { $sum: 1 } } },
  { $sort: { count: -1 } },
  { $limit: 10 },
];

// Get pipeline statistics
const stats = PipelineUtils.getStats(pipeline);
console.log(stats);
// Output:
// {
//   stageCount: 4,
//   stageTypes: { '$match': 1, '$group': 1, '$sort': 1, '$limit': 1 },
//   totalSize: 156,
//   estimatedComplexity: 6
// }

// Get human-readable description
const description = PipelineUtils.describe(pipeline);
console.log(description);

Pipeline Manipulation

// Clone a pipeline
const clonedPipeline = PipelineUtils.clone(pipeline);

// Filter stages by type
const matchStages = PipelineUtils.filterByStageType(pipeline, "$match");

// Remove specific stage type
const pipelineWithoutLimit = PipelineUtils.removeStageType(pipeline, "$limit");

// Insert a stage at specific position
const newPipeline = PipelineUtils.insertStage(pipeline, 1, {
  $addFields: { processed: true },
});

// Replace a stage
const modifiedPipeline = PipelineUtils.replaceStage(pipeline, 0, {
  $match: { status: "inactive" },
});

Pipeline Comparison

const pipeline1 = [
  { $match: { status: "active" } },
  { $group: { _id: "$category", count: { $sum: 1 } } },
];

const pipeline2 = [
  { $match: { status: "active" } },
  { $group: { _id: "$category", count: { $sum: 1 } } },
  { $sort: { count: -1 } },
];

const comparison = PipelineUtils.compare(pipeline1, pipeline2);
console.log(comparison);

Advanced Usage

Creating Reusable Pipeline Components

import { PipelineBuilder, PipelineStage } from "mongo-pipeline-kit";

// Define a reusable pipeline component
const activeUsersStage: PipelineStage = {
  $match: {
    status: "active",
    lastLogin: { $gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) },
  },
};

// Use the component in multiple pipelines
const userStatsPipeline = builder
  .addStage(activeUsersStage)
  .group({
    _id: "$role",
    count: { $sum: 1 },
  })
  .build();

Pipeline Composition

import { PipelineBuilder, PipelineComposer } from "mongo-pipeline-kit";

const composer = new PipelineComposer();

// Create separate pipeline segments
const filteringPipeline = builder.match({ status: "active" }).build();

const aggregationPipeline = builder
  .group({
    _id: "$category",
    total: { $sum: "$amount" },
  })
  .build();

// Compose pipelines
const finalPipeline = composer
  .compose(filteringPipeline, aggregationPipeline)
  .build();

// Export composed pipeline with metadata
const exportData = composer.exportWithMetadata({
  description: "Composed pipeline for user analytics",
  tags: ["analytics", "users"],
});

API Documentation

PipelineBuilder

The main class for building MongoDB aggregation pipelines.

Core Methods

• addFields(fields: object): Add a $addFields stage
• addStage(stage: PipelineStage): Add a custom pipeline stage
• bucket(options: object): Add a $bucket stage
• bucketAuto(options: object): Add a $bucketAuto stage
• build(): Build the final pipeline array
• collStats(options: object): Add a $collStats stage
• count(fieldName: string): Add a $count stage
• facet(options: object): Add a $facet stage
• geoNear(options: object): Add a $geoNear stage
• graphLookup(options: object): Add a $graphLookup stage
• group(expression: object): Add a $group stage
• indexStats(options: object): Add a $indexStats stage
• limit(n: number): Add a $limit stage
• lookup(options: object): Add a $lookup stage
• match(condition: object): Add a $match stage
• merge(options: object): Add a $merge stage
• out(collection: string | object): Add an $out stage
• project(projection: object): Add a $project stage
• redact(expression: object): Add a $redact stage
• replaceRoot(newRoot: object): Add a $replaceRoot stage
• replaceWith(newRoot: object): Add a $replaceWith stage
• sample(options: { size: number }): Add a $sample stage
• setWindowFields(options: object): Add a $setWindowFields stage
• skip(n: number): Add a $skip stage
• sort(sort: object): Add a $sort stage
• sortByCount(expression: any): Add a $sortByCount stage
• unionWith(options: string | object): Add a $unionWith stage
• unset(fields: string | string[]): Add an $unset stage
• unwind(field: string | object): Add a $unwind stage

New JSON Export Methods

• toJSON(pretty?: boolean): Convert pipeline to JSON string
• toObject(): Get pipeline as plain object
• exportWithMetadata(metadata): Export pipeline with additional metadata
• toString(): Get human-readable string representation

Pipeline Management Methods

• clear(): Clear all stages from the pipeline
• getStageCount(): Get the current number of stages
• getStage(index: number): Get a specific stage
• replaceStage(index: number, stage: PipelineStage): Replace a stage
• removeStage(index: number): Remove a stage

PipelineComposer

Utility class for composing and reusing pipeline segments.

Methods

• compose(...pipelines: Pipeline[]): Compose multiple pipelines
• extend(basePipeline: Pipeline, extension: Pipeline): Extend an existing pipeline
• validate(pipeline: Pipeline): Validate a pipeline structure
• toJSON(pretty?: boolean): Convert composed pipeline to JSON
• toObject(): Get composed pipeline as plain object
• exportWithMetadata(metadata): Export with metadata
• toString(): Get human-readable representation
• hasStage(operator: string): Check if pipeline contains specific stage type
• getStagesByType(operator: string): Get all stages of a specific type

PipelineUtils

Static utility class for pipeline operations and analysis.

Methods

• fromJSON(jsonString: string): Parse JSON string to pipeline
• toJSON(pipeline: Pipeline, pretty?: boolean): Convert pipeline to JSON
• validate(pipeline: Pipeline, options?): Validate pipeline structure
• clone(pipeline: Pipeline): Deep copy pipeline
• getStats(pipeline: Pipeline): Get pipeline statistics
• describe(pipeline: Pipeline): Get human-readable description
• compare(pipeline1: Pipeline, pipeline2: Pipeline): Compare two pipelines
• filterByStageType(pipeline: Pipeline, operator: string): Filter stages by type
• removeStageType(pipeline: Pipeline, operator: string): Remove stages by type
• insertStage(pipeline: Pipeline, index: number, stage: PipelineStage): Insert stage
• replaceStage(pipeline: Pipeline, index: number, stage: PipelineStage): Replace stage

Examples

For comprehensive examples of all features, see EXAMPLES.md.

🐛 Issues & Support

Getting Help

If you encounter any issues or need help with mongo-pipeline-kit:

• 📚 Check Documentation: Review the README.md and EXAMPLES.md
• 🔍 Search Issues: Look for similar issues in GitHub Issues
• 🐛 Report Bugs: Create a bug report using our issue template
• 🚀 Request Features: Suggest new features using our feature request template
• ❓ Ask Questions: Use our question template for help
• 📧 Email Support: Contact [e-mail] for private matters

Issue Guidelines

• Use Templates: We provide issue templates for bugs, features, and questions
• Include Details: Provide environment info, code examples, and error messages
• Be Specific: Describe what you're trying to achieve and what went wrong
• Search First: Check existing issues to avoid duplicates

For detailed information about reporting issues, see ISSUES.md.

Contributing

We welcome contributions! Please see our Contributing Guide for details on how to get started.

Publishing

For information about publishing new versions, please refer to our Publishing Guide.

License

MIT

Changelog

[0.3.3] - 2024-01-15

Added

• GitHub Sponsors Integration: Now fully active and working at https://github.com/sponsors/arjun-computer-geek
• FUNDING.md: Updated funding page with active sponsorship link
• Sponsorship Benefits: Clear explanation of how sponsorships help the project

Changed

• Updated package.json with working GitHub Sponsors funding link
• Enhanced FUNDING.md with active sponsorship information
• Improved funding configuration for better user experience

Fixed

• Funding Link: Now properly redirects to active GitHub Sponsors page
• Sponsorship Flow: Users can now sponsor the project directly through GitHub