mongoose-timeseries

mongoose-timeseries

a time series data and analytics storage plugin for Mongoose.

Installation

npm install mongoose-timeseries

Usage

Create your Schema

var YourDocumentSchema = new Schema({
   attr1: { type: Schema.Types.ObjectId, ref: 'attr1' },
   attr2: { type: Schema.Types.ObjectId, ref: 'attr2' },
   date: { type: Date, default: Date.now },
   analytics: {
     metric: { type: Number }
   },
   info: {
      sub1: { type: String },
      sub2: { type: String },
      sub3: { type: String }
   }
})

Apply Plugin

var mongoose = require('mongoose')
var timeseries = require('mongoose-timeseries')

YourDocumentSchema.plugin(timeseries, {
   target: 'TimeSeriesDocument',
   dateField: 'date',
   resolutions: ['minute', 'day'],
   key: {
      attr1: 1,
      attr2: 1,
      info: function(doc) {
         return doc.info.sub1 + doc.info.sub2 + doc.info.sub3
      }
   },
   data: {
      metric: {
         source: 'analytics.metric',
         operations: ['sum', 'max', 'min'],
         calculations: ['average', 'range', 'range_min', 'range_max']
      }
   }
})

Watch your time series data grow!

Your saved time series documents will look like:

{
  date: {
    start: Mon Aug 01 2016 00:00:00 GMT-0600(MDT),
    end: Mon Aug 01 2016 23:58:42 GMT-0600(MDT)
  }
  resolution: 'day',
  count: 5,
  data: {
    metric: {
      count: 5,
      sum: 697,
      min: 100,
      max: 200
    }
  },
  key: {
    attr1: '55931aba4f3b26d63810a55d',
    attr2: '5536011b00a57af8243d7e5b',
    info: 'ABC'
  },
  _id: 57 a50178e47cea6f5d7f1c3b
}

Your queried and found time series documents will look like:

{
  date: {
    start: Mon Aug 01 2016 00:00:00 GMT-0600(MDT),
    end: Mon Aug 01 2016 23:58:42 GMT-0600(MDT)
  }
  resolution: 'day',
  count: 5,
  data: {
    metric: {
      count: 5,
      sum: 697,
      min: 100,
      max: 200,
      average: 139.4,
      range: 100,
      range_min: 39.4,
      range_max: 60.6
    }
  },
  key: {
    attr1: '55931aba4f3b26d63810a55d',
    attr2: '5536011b00a57af8243d7e5b',
    info: 'ABC'
  },
  _id: 57 a50178e47cea6f5d7f1c3b
}

This is because the calculations are performed as middleware during Mongoose Find() executions.

Documentation

Function

YourDocumentSchema.plugin(timeseries, options(Object))

You can apply multiple times with different options:

YourDocumentSchema.plugin(timeseries, options1(Object))
YourDocumentSchema.plugin(timeseries, options2(Object))
YourDocumentSchema.plugin(timeseries, options3(Object))

Options

target(String)

The MongoDB collection name (destination) of the specific time series data.

---

dateField(String)

The custom date field of your schema (if applicable). If not set, defaults to document._id.getTimestamp()

---

resolutions(Array)

The time series resolutions you want: Can include any or all of ['minute', 'hour', 'day', 'month']

---

key(Object)

The unique information you'd like your time series to separate and store.

key.'attribute'(Number | Function)

For each key, use the number '1' to relay the name, or a function that returns your value to store on the key.

---

data(Object)

The data you'd like to keep track of.

data.'attribute'.source(String)

The source of the parameter you're tracking. Can be nested like:

'analytics.metrics.metric1'

data.'attribute'.operations(Array of Strings)

The operations to perform. Currently supports 'sum', 'max', and 'min'

data.'attribute'.calculations(Array of Strings)

The "post-find" calculations to perform. Currently supports 'average', 'range', 'range_min', and 'range_max'

average = sum / count, range = max - min, range_min = average - min, (must also include average), range_max = max - average (must also include average)

---

Using the Time Series Data

Now, in your front-end analytics, you can query the time series data like:

var startDateFromUI = ...
var endDateFromUI = ...

TimeSeriesAnalyticsModel.find({
  resolution: 'day',
  'date.start': {
    $gte: startDateFromUI,
    $lte: endDateFromUI
  }
} function(err, results) {
  console.log(results)
})

Tests (incomplete)

npm install
npm test

Assumptions

• Original source documents are a continual stream of data being dumped
• Documents in the source time-series collection are never themselves found and updated

To-do

• Tests
• Auto-indexing
• Auto-remove (removes source time-series documents automatically after a set interval... capped collection?)

License

MIT