metrics-broker

metrics-broker: Comprehensive instrumentation for your Node programs

metrics-broker helps you easily instrument your applications and make the results available to your monitoring system.

The broker and instrument modules provided by this package make it easy for your programs to record events and essential timing information to a host-level metrics store that keeps track of the values on their behalf. This data can then be easily collected by a monitoring agent and then further handled by your metrics collection system (for alerting, aggregation, graphing, etc.).

The separation of generation and storage of metrics has many benefits:

• It helps ensure that the data will be maintained even if the program that generates them is terminated.

• It simplifies consuming metrics by reducing the number of collection points. Without such a broker, each Node.js process would have to be separately queried for metric data on separate ports, adding significant complexity to the collection process.

• It reduces aggregation burden. If multiple Node.js processes on a host are all running the same module and recording the same metrics, the broker will automatically aggregate the metrics on a per-host basis before the data gets into the monitoring system, which is usually what you want.

Terminology

Each metric has two important attributes: a name and zero or more dimensions.

The name is an identifier for the metric. An example would be "requests" or "request_latency". This name should be the same name as what would appear in the Y-axis legend of a graph.

Dimensions are a somewhat trickier concept. They are schemaless keyword arguments that add context to a metric. It may help to think of dimensions as a set of predicates against which you might filter the metrics when choosing what to graph. Some simple examples would be { host: 'www1'}, { cust: 'AcmeWidgets' }, or { disk: 'sda' }. Of course, you're not limited to just one dimension; you can combine them as well: { host: 'www1', cust: 'AcmeWidgets' }.

• Tip: Use uniform dimension key names for a given metric name. In other words, every time you record data for metric name 'foo', if you ever associate a dimension name 'bar' with it, do it all the time.*

Types of metrics

This module supports the following types of metrics:

• Counter: A variable whose value you increment each time a relevant event occurs. (Note that due to JavaScript's implementation of numeric values, a counter will wrap around to 0 when it reaches 2^32.)

• Histogram: A container for tracking value distributions. The object will automatically maintain statistical information (min, max, sum, variance, standard deviation, mean, median, etc.) for you. (Note that this is not really a traditional histogram as it is not implemented in terms of buckets with fixed boundaries. A traditional histogram type may be implemented in a future release.)

• Timer: A histogram variant that tracks timing information for you. This is especially useful for tracking latencies.

Instruments

Instruments are the primary means by which a program should record metrics. These are very lightweight functions that communicate with the broker, which keeps track of the actual values.

Example usage:

var Instruments = require('metrics-broker').Instruments;
var Counter = Instruments.Counter;
var Histogram = Instruments.Histogram;
var Timer = Instruments.Timer;

// increment a counter (note: the dimensions may be null if not needed)
Counter.inc('requests', { cust: 'AcmeWidgets' });

// record latency
var t = new Timer('db_query_latency', { db: 'custDB' });
t.start();
// after the response has been received:
t.stop();
// the timer object can be reused; just call start() again.

// Update a histogram (rarely used directly)
Histogram.update('observed', { city: 'San Francisco' });

Metrics broker

The metrics broker maintains metrics on behalf of the entire host. When the instruments modules are used, each Node.js instance will automatically pass the metric change (update, increment, etc.) to the broker via a UNIX datagram socket.

The broker is intended to be queried by a monitoring agent that can forward the metrics data to a central monitoring system. To that end, it provides a simple HTTP server on port 22222 that will return the current metric values in JSON format (just issue it a GET request for /metrics).

The broker can be launched via npm start metrics-broker. It does not detach itself from the terminal and logs to stdout/stderr, making it easy to wrap inside daemontools, runit, or other process supervisor systems.

Acknowledgments

Thanks to Mike Ihbe for the metrics module, which provided a good foundation upon which to build this.