flow-rolling-stats

Rolling Window Stats

This Javascript library provides rolling time series operators for unevenly spaced data, such as simple moving averages (SMAs), exponential moving averages (EMAs), and various rolling functions. It is based on the utsAlgorithms C library.

The implementation details are described in "Algorithms for Unevenly Spaced Time Series: Moving Averages and Other Rolling Operators", Eckner (2017).

Usage

npm install flow-rolling-stats

NPM scripts

• npm test: Run test suite
• npm start: Run npm run build in watch mode
• npm run test:watch: Run test suite in interactive watch mode
• npm run test:prod: Run linting and generate coverage
• npm run build: Generate bundles and typings, create docs
• npm run lint: Lints code
• npm run commit: Commit using conventional commit style (husky will tell you to use it if you haven't)

API

Operators are implemented as Javascript classes with a standard interface and designed to be used in real-time processing pipeline where observations are received periodically at uneven time intervals.

Typical usage is shown below (where RollingMean can be replaced with any of the supported operator classes):

const windowedAvg1 = new RollingMean({
  windowSize: { duration: 15, unit: 'minutes' }
});

const windowedAvg2 = new RollingMean({
  windowSize: { duration: 15, unit: 'minutes' }
});

// Process an event as it arrives
on('event', event => {

  // Construct metric
  const metric = {
    timestamp: event.timestamp;
    avgValue1: windowedAvg1.update(event.value1, event.timestamp);
    avgValue2: windowedAvg2.update(event.value2, event.timestamp);
  };

  // Send metric to next stage of pipeline
  emit(metric);
});

Operators

Class	Operation	Comments
RollingCount	Count of observations in window
RollingMin	Min value of observations in window
RollingMax	Max value of observations in window
RollingMean	Arithmetic mean of observations in window
RollingSum	Sum of observations in window
SmaLast	Simple moving average using last value
SmaNext	Simple moving average using next value
SmaLinear	Simple moving average using linear interpolation
EmaLast	Exponentially-weighted average using last value
EmaNext	Exponentially-weighted average using next value
EmaLinear	Exponentially-weighted average using linear interpolation

Window Size

Window size is specified when constructing an operator class. Supported time units are:

• ms
• milliseconds
• second(s) -minute(s)
• hour(s)
• days(s)

Recording Observations

Call the update method of an operation to record a new observation. Pass the observation value and timestamp (in epoch milliseconds) as parameters. The update method returns the calculated statistic after the window has been updated.

Note that value property returns the calculated statistic and can be called at any time without performing a window update.

Numerical Stability

Many of the operators maintain a sum that is updated over the lifetime of the operator. To maintain numerical stability and negate the effects of limited floating point precision the method of Kahan (1965)[1] is used to perform compensated addition.

[1] Kahan, W. (1965). Further remarks on reducing truncation errors Communications of the ACM 8 (1), 40.