spm-agent

Sematext Agent Framework for Node.js

A framework for monitoring agents. Ship metrics to Sematext Cloud or InfluxDB.

Functionality

• Sender interface to Sematext backend receivers
• Buffering metrics in case of network outages
• Reconnect after failures
• Logging functions
• Configuration handling
• Pluggable agents

Example to Implement a Monitoring Agent

Sematext Cloud supports the Influx Line Protocol for the metric ingestion.

Agent modules must provide a start and stopfunction.

The function agent.addMetrics(metric) collects a metric, which is shipped in bulk requests via Influx Line Protocol to the metrics receiver.

All metric objects, must have the mandatory fields measurement, tags and fields:

• measurement - the name of the measurement e.g. 'process.memry'
• tags - key/value pairs, useful for filtering or aggregation of data
• fields - numeric fields with the measurement values

var SpmAgent = require('spm-agent')
var client = new SpmAgent()
var os = require('os')

// configure client, with non-defautl values
// or use the file ./.spmagentrc in YAML format
// the default configuration contains values for Sematext Cloud US
client.config.tokens.spm = process.env.MONITORING_TOKEN
client.config.influx = {
  dbName: 'metrics',
  // change receiver to Sematext Cloud EU
  host: 'spm-receiver.eu.sematext.com',
  port: 443,
  protocol: 'https'
}

var testAgent = client.createAgent(new SpmAgent.Agent ({
  start: function (agent) {
    // initialize your metrics collector ...
    // Typically agents collect metrics periodically, every N seconds. The time between // two collection activities is the collectionInterval, specified in milliseconds.
    this.tid = setInterval(function () {
      // get every N seconds some metrics
      agent.addMetrics({
        measurement: 'process.memory',
        tags: {role: 'frontend', 'os.host': os.hostname()},
        fields: {rss: process.memoryUsage().rss}}
       )
      // The monitoring token can be set as 'tags.token' value
      // Routing a metrics to a different monitoring app
      // requires setting the `token` tag
      agent.addMetrics({
         measurement: 'process.memory',
         tags: {token: 'SOME_OTHER_APP_TOKEN_HERE'},
        fields: {rss: process.memoryUsage().rss}
      })
    }, client.config.collectionInterval)
  },
  stop: function () {
    if (this.tid) {
       cancelInterval(this.tid)
    }
  }
}))

// log collected metrics to console
// observe all metrics
testAgent.on('metrics', console.log)
// observe a specific metric using the measurment name
testAgent.on('process.memory', console.log)

Metric Tags

The agent is able to use environment variables as metrics tags. Define a list of environment variables to be used as metric tags.

Let's assume this is the enviroment of the monitored application:

> env
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/v6/bh2q1xsn27g4d2g54z2ylv100000gn/T/
TERM_PROGRAM_VERSION=404.1
TERM_SESSION_ID=F12E0DBD-BF40-466D-85EC-08E89EDC3440
USER=stefan
PWD=/Users/stefan
HOME=/Users/stefan
LOGNAME=stefan
LC_CTYPE=UTF-8
DISPLAY=/private/tmp/com.apple.launchd.J60ybGpban/org.macosforge.xquartz:0
_=/usr/bin/env

You want to see later the performance metrics aggregated or filtered for specific user. In this case the agent should collect the user name as tag. You can find the user name in the process environment variable USER. Let's also assume the workingdirectory of the application PWD is another relevant identifier for your monitored application.

To configure the agent collecting USER and PWD as tags you can specify a commaseparated list of environment variables for the the agent configuration:

# add the values of env. variables USER, PWD as tags to metrics
export MONITORING_TAGS_FROM_ENV="USER, PWD"

The generated metrics include then the environment variable name and its value as tags. The result in Influx Line Protocol (ILP) format, produced by the agent for metric called swap:

swap, USER=stefan,PWD=/Users/stefan out=0i,in=0i 1576765680000000000

Define static tags in key:value format

export MONITORING_TAGS_FROM_ENV="organisation:sematext, service:api"

The result in in ILP format:

swap, organisation=sematext,service=api out=0i,in=0i 1576765680000000000

Both types of tags can be mixed:

export MONITORING_TAGS_FROM_ENV="organisation:sematext, service:api, USER, PWD"

The result in in ILP format:

swap, organisation=sematext,service=api,USER=stefan,PWD=/Users/stefan,os.host=imac.local out=0i,in=0i 1576765680000000000

The config file entry influx.tagsFromEnv in .spmagenrc works as well:

tokens: 
  spm: 'YOUR_MONITORING_TOKEN'
  infra: 'YOUR_INFRA_MONITROING_TOKEN'
influx:
  tagsFromEnv: 'organisation:sematext, USER, PWD' 
  dbName: 'metrics'
  host: spm-receiver.sematext.com
  protocol: https
  port: 443
  

Contribute

Let us know about monitoring agents you need, maybe you like to contribute with your domain expertise!

Related Modules

• Node.js Monitoring
• spm-agent-nodejs,
• spm-agent-mongodb,
• sematext-agent-nginx,
• sematext-agent-httpd
• Sematext Cloud - one platform for metrics and logs