@gigasource/mongoose-sync

This module is a Mongoose middleware for syncing between 2 databases by replicating actions executed on the primary database

1. Features:

• Support syncing for the following actions:
  • Create
    • Model.create (single document)
    • Model.create (array of documents)
    • document.save
  • Update
    • Model.findOneAndUpdate
    • Model.findByIdAndUpdate (this uses 'findOneAndUpdate' hook)
    • Model.update (with option multi=false - default is false)
    • Model.updateOne
    • Model.updateMany
  • Delete
    • document.remove (Model.remove won't work with post remove middleware)
    • Model.remove (not supported by Mongoose so this module uses a workaround). Note: this use 'model-create' as the hook name
    • Model.findOneAndRemove
    • Model.findByIdAndRemove (this uses 'findOneAndRemove' hook)
• Job queue: actions are pushed into a job queue for execution.
• Sync interval: user can specify an interval for syncing. When the job queue has emptied all the tasks, the queue will be paused and resumed on the next interval.
• Retries: when a task fails, job queue will pause and retry the task on the next interval
• MongoDB storage for job queue: tasks are stored in a collection in the primary database

2. Tests:

Unit tests for retry logic is currently not implemented on 'master' branch because it requires generating random errors inside queue processing logic but it's inappropriate to put error-generating logic in production code

3. Usage:

The job queue is mandatory for the module to work so user must initialize the queue first using this function: (/src/middleware/index.js):

module.exports.init = async function (syncDbConnectionUrl, syncDbOptions, syncInterval, enableMongoStore) {
...
}

// Example:
const {init} = require('/lib');
await mongoose.connect('mongodb://localhost:27017/', {dbName}); // this must be called first
await init('mongodb://localhost:27017/', {dbName, user, pass}, 15 * 60 * 1000); // enableMongoStore = true by default

It is required that user calls this after mongoose.connect to primary db and before creating Mongoose schemas and models

Now apply the middleware to Mongoose Schema:

const {applySyncMiddleware} = require('/lib');
const carSchema = new Schema({
  model: {
    type: String,
    trim: true,
  },
  year: Number,
  weight: Number,
});

const saveSyncCondition = function (car) {
  return car.year > 2000;
};

const updateSyncCondition = function (updatedCar) {
  return updatedCar.weight > 100;
};

// Add middleware here
// conditions are functions returning true or false
applySyncMiddleware(carSchema, {
  create: saveSyncCondition, // condition for actions related to 'create'
  update: updateSyncCondition, // condition for actions related to 'update'
});

// applySyncMiddleware(carSchema, syncCondition); // if 1 function is passed, it is used for both 'create' and 'update'
// -------------------

const model = mongoose.model(schemaName, carSchema);