aws-xray-sdk

What is aws-xray-sdk?

The aws-xray-sdk is an npm package that provides tools for integrating AWS X-Ray tracing into your Node.js applications. AWS X-Ray helps developers analyze and debug distributed applications, such as those built using a microservices architecture. The SDK allows you to trace requests as they travel through your application, providing insights into performance bottlenecks and errors.

What are aws-xray-sdk's main functionalities?

Automatic Instrumentation

This feature allows you to automatically capture HTTP and HTTPS requests made by your application. The SDK wraps the native HTTP and HTTPS modules to capture and record data about outgoing requests.

const AWSXRay = require('aws-xray-sdk');
const http = AWSXRay.captureHTTPs(require('http'));
const https = AWSXRay.captureHTTPs(require('https'));

http.get('http://www.example.com', (res) => {
  res.on('data', (chunk) => {
    console.log(`BODY: ${chunk}`);
  });
  res.on('end', () => {
    console.log('No more data in response.');
  });
});

Manual Instrumentation

This feature allows you to manually create and manage subsegments within your code. You can add annotations and metadata to these subsegments to provide additional context for your traces.

const AWSXRay = require('aws-xray-sdk');

AWSXRay.captureFunc('annotations', function(subsegment) {
  subsegment.addAnnotation('userId', '12345');
  subsegment.addMetadata('key', 'value');
});

Middleware Integration

This feature allows you to integrate AWS X-Ray with popular web frameworks like Express. The SDK provides middleware that automatically creates and closes segments for incoming HTTP requests.

const AWSXRay = require('aws-xray-sdk');
const express = require('express');
const app = express();

app.use(AWSXRay.express.openSegment('MyApp'));

app.get('/', function (req, res) {
  res.send('Hello World!');
});

app.use(AWSXRay.express.closeSegment());

app.listen(3000);

Other packages similar to aws-xray-sdk

jaeger-client

Jaeger is an open-source end-to-end distributed tracing tool originally developed by Uber Technologies. The jaeger-client npm package allows you to instrument your Node.js applications to send trace data to a Jaeger backend. Unlike aws-xray-sdk, Jaeger is designed to work with the Jaeger tracing system and provides more flexibility in terms of deployment and configuration.

README

Setup development environment

npm install

Run tests

npm test

Generate docs

grunt docs

Requirements

AWS SDK v2.7.2 or greater.

AWS X-Ray

AWS X-Ray automatically records information for incoming and outgoing requests and responses, as well as local data such as function calls, time, variables (via metadata and annotations), even EC2 instance data. Currently only supports Express applications for auto capturing.

The AWS X-Ray SDK has two modes - Manual and CLS. CLS mode uses the Continuation Local Storage package and automatically keeps track of the current segment and subsegment. This is the default mode. Manual mode requires you pass around the segment reference.

In CLS mode, you can get the current segment/subsegment at any time: var segment = AWSXRay.getSegment();

In manual mode, you can get the base segment off the request object: var segment = req.segment;

• In manual mode, you must pass along the reference to the current subsegment.
• If you are not using the express middleware, and you create your segment manually, you must close it with segment.close().

The SDK exposes the Segment and Subsegment objects to create your own capturing mechanisms, but a few are supplied. These keep the current subsegment up to date automatically using CLS.

AWSXRay.capture - Takes a function that takes a single subsegment argument. This will create a new nested subsegment and expose it. The segment will close automatically when the function completes executing. This will not corretly time functions with asynchronous calls, instead use captureAsync.

AWSXRay.captureAsync - Takes a function that takes a single subsegment argument. This will create a new nested subsegment and expose it. The segment must be closed manually using subsegment.close() or subsegment.close(error) when the asynchronous function completes.

AWSXRay.captureCallback - Takes a function to be used as a callback. Useful for capturing callback information and directly associate it to the call that generated it. This will create a new nested subsegment and expose it by appending it onto the arguments used to call the callback. For this reason, always call your captured callbacks with the full parameter list. The subsegment will close automatically when the function completes executing.

Setup

environment variables

XRAY_DEBUG_MODE                  Enables logging to console output (otherwise outputs to AWSXRay.log).
XRAY_TRACING_NAME                *For setting the default segment name. Internal use only. Ex: An app running on AWS Elastic Beanstalk will have this set automatically.
XRAY_TRACING_DEFAULT_NAME        *For setting the default segment name. This does not override XRAY_TRACING_NAME, but does override setting it via code.

POSTGRES_DATABASE_VERSION        Sets additional data for the sql subsegment.
POSTGRES_DRIVER_VERSION          Sets additional data for the sql subsegment.

MYSQL_DATABASE_VERSION           Sets additional data for the sql subsegment.
MYSQL_DRIVER_VERSION             Sets additional data for the sql subsegment.

Segment names default to XRAY_TRACING_NAME if set, otherwise, req.headers 'host' field will be used. If this field is missing or an IP (in the case of an application load balancer), then XRAY_TRACING_DEFAULT_NAME will be used if available. If neither of these environment variables is set, it will check the value set in code. If it is not set in code, an error will be thrown.

version capturing

Use the 'npm start' script to enable.

capture all incoming HTTP requests to '/'

var app = express();

//...

var AWSXRay = require('aws-xray-sdk');
var AWSXRay.setDefaultName('myDefaultSegment');           //required if XRAY_TRACING_DEFAULT_NAME is not set

app.use(AWSXRay.express.openSegment());                   //required at the start of your routes

app.get('/', function (req, res) {
  res.render('index');
});

app.use(AWSXRay.express.closeSegment());   //required at the end of your routes / first in error handling routes

capture all outgoing AWS requests

var AWS = captureAWS(require('aws-sdk'));

//create new clients as per usual
//make sure any outgoing calls that are dependent on another async
//function are wrapped with captureAsync, otherwise duplicate segments may leak
//see usages for clients in manual and CLS modes

configure AWSXRay to automatically capture EC2 instance data

var AWSXRay = require('aws-xray-sdk');
AWSXRay.config([AWSXRay.plugins.EC2]);

add annotations

var key = 'hello';
var value = 'there';        // must be string, boolean or finite number

subsegment.addAnnotation(key, value);

add metadata

var key = 'hello';
var value = 'there';

subsegment.addMetadata(key, value);

create new subsegment

var newSubseg = subsegment.addNewSubsegment(name);

// or

var subsegment = new Subsegment(name);

CLS Mode Examples

capture all incoming HTTP requests to '/'

var app = express();

//...

var AWSXRay = require('aws-xray-sdk');

app.use(AWSXRay.express.openSegment());

app.get('/', function (req, res) {
  res.render('index');
});

app.use(AWSXRay.express.closeSegment());

capture through function calls

var AWSXRay = require('aws-xray-sdk');

app.use(AWSXRay.express.openSegment());

app.get('/', function (req, res) {
  var host = 'samplego-env.us-east-1.elasticbeanstalk.com';

  AWSXRay.captureAsync('send', function(seg) {
    sendRequest(host, function() {
      console.log("rendering!");
      res.render('index');
      seg.close();
    });
  });
});

app.use(AWSXRay.express.closeSegment());

function sendRequest(host, cb) {
  var options = {
    host: host,
    path: '/',
  };

  var callback = function(response) {
    var str = '';

    response.on('data', function (chunk) {
      str += chunk;
    });

    response.on('end', function () {
      cb();
    });
  }

  http.request(options, callback).end();
};

capture outgoing AWS requests on a single client

var s3 = AWSXRay.captureAWSClient(new AWS.S3());

//use client as usual
//make sure any outgoing calls that are dependent on another async
//function are wrapped with captureAsync, otherwise duplicate segments may leak

capture outgoing AWS requests on every AWS SDK client

var aws = AWSXRay.captureAWS(require('aws-sdk'));

//create new clients as per usual
//make sure any outgoing calls that are dependent on another async
//function are wrapped with captureAsync, otherwise duplicate segments may leak

capture all outgoing HTTP/S requests

AWSXRay.captureHTTPs(http);     //patching the http module will patch for https as well

var options = {
  ...
}

http.request(options, callback).end();

//create new requests as per usual
//make sure any outgoing calls that are dependent on another async
//function are wrapped with captureAsync, otherwise duplicate segments may leak

capture PostgreSQL queries

var AWSXRay = require('aws-xray-sdk');
var pg = AWSXRay.capturePostgres(require('pg'));

...

var client = new pg.Client();

client.connect(function (err) {
  ...

  client.query({name: 'moop', text: 'SELECT $1::text as name'}, ['brianc'], function (err, result) {
    //automatically captures query and error (if any)
  });
});

...

var pool = new pg.Pool(config);
pool.connect(function(err, client, done) {
  if(err) {
    return console.error('error fetching client from pool', err);
  }
  var query = client.query('SELECT * FROM mytable', function(err, result) {
    //automatically captures query and error (if any)
  });
});

capture MySQL queries

var AWSXRay = require('aws-xray-sdk');
var mysql = AWSXRay.captureMySQL(require('mysql'));

var config = { ... };

...

var connection = mysql.createConnection(config);

connection.query('SELECT * FROM cats', function(err, rows) {
  //automatically captures query and error (if any)
});

...

var pool  = mysql.createPool(config);
var segment = req.segment;

pool.query('SELECT * FROM cats', function(err, rows, fields) {
  //automatically captures query and error (if any)
}

Manual Mode

Enable manual mode:

AWSXRay.enableManualMode();

capture through function calls

var AWSXRay = require('aws-xray-sdk');

AWSXRay.enableManualMode();

app.use(AWSXRay.express.openSegment());

app.get('/', function (req, res) {
  var segment = req.segment;
  var host = 'samplego-env.us-east-1.elasticbeanstalk.com';

  AWSXRay.captureAsync('send', function(seg) {
    sendRequest(host, function() {
      console.log("rendering!");
      res.render('index');
      seg.close();
    });
  }, segment);
});

app.use(AWSXRay.express.closeSegment());

function sendRequest(host, cb, segment) {
  var options = {
    host: host,
    path: '/',
    Segment: segment
  };

  var callback = function(response) {
    var str = '';

    //the whole response has been recieved, so we just print it out here
    //another chunk of data has been recieved, so append it to `str`
    response.on('data', function (chunk) {
      str += chunk;
    });

    response.on('end', function () {
      cb();
    });
  }

  http.request(options, callback).end();
};

capture outgoing AWS requests on a single client

var s3 = AWSXRay.captureAWSClient(new AWS.S3());
var params = {
  Bucket: bucketName,
  Key: keyName,
  Body: 'Hello!',
  Segment: subsegment             //required "Segment" param
};

s3.putObject(params, function(err, data) {
  ...
});

capture all outgoing AWS requests

var AWS = captureAWS(require('aws-sdk'));

//create new clients as per usual
//make sure any outgoing calls that are dependent on another async
//functions are wrapped, otherwise duplicate segments may leak.

capture all outgoing HTTP/S requests

AWSXRay.captureHTTPs(http);     //patching the http module will patch for https as well

...

//include segment/subsegment reference in options as 'Segment'
var options = {
  ...
  Segment: subsegment
}

http.request(options, callback).end();

capture PostgreSQL queries

var AWSXRay = require('aws-xray-sdk');
var pg = AWSXRay.capturePostgres(require('pg'));

...

var client = new pg.Client();

client.connect(function (err) {
  ...

  client.query({name: 'moop', text: 'SELECT $1::text as name'}, ['brianc'], function (err, result) {
    //automatically captures query and error (if any)
  });
});

...

var pool = new pg.Pool(config);
pool.connect(function(err, client, done) {
  if(err) {
    return console.error('error fetching client from pool', err);
  }
  var query = client.query('SELECT * FROM mytable', function(err, result) {
    //automatically captures query and error (if any)
  }, segment));
};

capture MySQL queries

var AWSXRay = require('aws-xray-sdk');
var mysql = AWSXRay.captureMySQL(require('mysql'));

var config = { ... };

...

var connection = mysql.createConnection(config);

connection.query('SELECT * FROM cats', function(err, rows) {
  //automatically captures query and error (if any)
});

...

var pool  = mysql.createPool(config);

pool.query('SELECT * FROM cats', function(err, rows, fields) {
  //automatically captures query and error (if any)
}, segment);