express-http-proxy-async

express-http-proxy-async

Express middleware to proxy request to another host and pass response back to original caller.

This repository has been forked from https://github.com/villadora/express-http-proxy. Currently, the main difference between the two repositories is that this fork supports asynchronous promised based resolution of the host.

NOTE: version 1.0.0 released: breaking changes, transition guide at bottom of doc.

Install

$ npm install express-http-proxy-async --save

Usage

proxy(host, options);

Example:

To proxy URLS starting with '/proxy' to the host 'www.google.com':

var proxy = require('express-http-proxy-async');
var app = require('express')();

app.use('/proxy', proxy('www.google.com'));

Promises

Many function hooks support Promises.
If any Promise is rejected, next(x) is called in the hosting application, where x is whatever you pass to Promise.reject;

e.g.

  app.use(proxy('/reject-promise', {
    proxyReqOptDecorator: function() {
      return Promise.reject('An arbitrary rejection message.');
    }
  }));

eventually calls

next('An arbitrary rejection messasage');

Options

proxyReqPathResolver (supports Promises)

Provide a proxyReqPathResolver function if you'd like to operate on the path before issuing the proxy request. Use a Promise for async operations.

app.use('/proxy', proxy('localhost:12345', {
  proxyReqPathResolver: function(req) {
    return require('url').parse(req.url).path;
  }
}));

Promise form

app.use('/proxy', proxy('localhost:12345', {
  proxyReqPathResolver: function(req) {
    return new Promise(function (resolve, reject) {
      setTimeout(function () {   // simulate async
        var resolvedPathValue = "http://google.com";
        resolve(resolvedPathValue);
      }, 200);
    });
  }
}));

forwardPath

DEPRECATED. See proxyReqPathResolver

forwardPathAsync

DEPRECATED. See proxyReqPathResolver

filter

The filter option can be used to limit what requests are proxied. Return true to execute proxy.

For example, if you only want to proxy get request:

app.use('/proxy', proxy('www.google.com', {
  filter: function(req, res) {
     return req.method == 'GET';
  }
}));

userResDecorator (was: intercept) (supports Promise)

You can modify the proxy's response before sending it to the client.

app.use('/proxy', proxy('www.google.com', {
  userResDecorator: function(proxyRes, proxyResData, userReq, userRes) {
    data = JSON.parse(proxyResData.toString('utf8'));
    data.newProperty = 'exciting data';
    return JSON.stringify(data);
  }
}));

app.use(proxy('httpbin.org', {
  userResDecorator: function(proxyRes, proxyResData) {
    return new Promise(function(resolve) {
      proxyResData.funkyMessage = 'oi io oo ii';
      setTimeout(function() {
        resolve(proxyResData);
      }, 200);
    });
  }
}));

304 - Not Modified

When your proxied service returns 304, not modified, this step will be skipped, since there is no body to decorate.

exploiting references

The intent is that this be used to modify the proxy response data only.

Note: The other arguments (proxyRes, userReq, userRes) are passed by reference, so you can currently exploit this to modify either response's headers, for instance, but this is not a reliable interface. I expect to close this exploit in a future release, while providing an additional hook for mutating the userRes before sending.

gzip responses

If your proxy response is gzipped, this program will automatically unzip it before passing to your function, then zip it back up before piping it to the user response. There is currently no way to short-circuit this behavior.

limit

This sets the body size limit (default: 1mb). If the body size is larger than the specified (or default) limit, a 413 Request Entity Too Large error will be returned. See bytes.js for a list of supported formats.

app.use('/proxy', proxy('www.google.com', {
  limit: '5mb'
}));

memoizeHost

Defaults to true.

When true, the host argument will be parsed on first request, and memoized for subsequent requests.

When false, host argument will be parsed on each request.

E.g.,

  function coinToss() { return Math.random() > .5 }
  function getHost() {
    var host = coinToss() ? 'http://yahoo.com' : 'http://google.com';
    return coinToss() ? Promise.resolve(host) : host; // <-- Asynchronous using promises
  }
  
  app.use(proxy(getHost, {
    memoizeHost: false
  }))

In this example, when memoizeHost:false, the coinToss occurs on each request, and each request could get either value.

Conversely, When memoizeHost:true, the coinToss would occur on the first request, and all additional requests would return the value resolved on the first request.

userResHeaderDecorator

app.use('/proxy', proxy('www.google.com', {
  userResHeaderDecorator(headers, userReq, userRes, proxyReq, proxyRes) {
    // recieves an Object of headers, returns an Object of headers.
    return headers;
  }
}));

decorateRequest

REMOVED: See proxyReqOptDecorator and proxyReqBodyDecorator.

skipToNextHandlerFilter(supports Promise form)

(experimental: this interface may change in upcoming versions)

Allows you to inspect the proxy response, and decide if you want to continue processing (via express-http-proxy-async) or call next() to return control to express.

app.use('/proxy', proxy('www.google.com', {
  skipToNextHandlerFilter: function(proxyRes) {
    return proxyRes.statusCode === 404;
  }
}));

proxyReqOptDecorator (supports Promise form)

You can override most request options before issuing the proxyRequest. proxyReqOpt represents the options argument passed to the (http|https).request module.

NOTE: req.path cannot be changed via this method; use proxyReqPathResolver instead. (see https://github.com/villadora/express-http-proxy/issues/243)

app.use('/proxy', proxy('www.google.com', {
  proxyReqOptDecorator: function(proxyReqOpts, srcReq) {
    // you can update headers
    proxyReqOpts.headers['Content-Type'] = 'text/html';
    // you can change the method
    proxyReqOpts.method = 'GET';
    return proxyReqOpts;
  }
}));

You can use a Promise for async style.

app.use('/proxy', proxy('www.google.com', {
  proxyReqOptDecorator: function(proxyReqOpts, srcReq) {
    return new Promise(function(resolve, reject) {
      proxyReqOpts.headers['Content-Type'] = 'text/html';
      resolve(proxyReqOpts);
    })
  }
}));

proxyReqBodyDecorator (supports Promise form)

You can mutate the body content before sending the proxyRequest.

app.use('/proxy', proxy('www.google.com', {
  proxyReqBodyDecorator: function(bodyContent, srcReq) {
    return bodyContent.split('').reverse().join('');
  }
}));

You can use a Promise for async style.

app.use('/proxy', proxy('www.google.com', {
  proxyReqBodyDecorator: function(proxyReq, srcReq) {
    return new Promise(function(resolve, reject) {
      http.get('http://dev/null', function (err, res) {
        if (err) { reject(err); }
        resolve(res);
      });
    })
  }
}));

https

Normally, your proxy request will be made on the same protocol as the original request. If you'd like to force the proxy request to be https, use this option.

app.use('/proxy', proxy('www.google.com', {
  https: true
}));

preserveHostHdr

You can copy the host HTTP header to the proxied express server using the preserveHostHdr option.

app.use('/proxy', proxy('www.google.com', {
  preserveHostHdr: true
}));

parseReqBody

The parseReqBody option allows you to control parsing the request body. For example, disabling body parsing is useful for large uploads where it would be inefficient to hold the data in memory.

This defaults to true in order to preserve legacy behavior.

When false, no action will be taken on the body and accordingly req.body will no longer be set.

Note that setting this to false overrides reqAsBuffer and reqBodyEncoding below.

app.use('/proxy', proxy('www.google.com', {
  parseReqBody: false
}));

reqAsBuffer

Note: this is an experimental feature. ymmv

The reqAsBuffer option allows you to ensure the req body is encoded as a Node Buffer when sending a proxied request. Any value for this is truthy.

This defaults to to false in order to preserve legacy behavior. Note that the value of reqBodyEnconding is used as the encoding when coercing strings (and stringified JSON) to Buffer.

Ignored if parseReqBody is set to false.

app.use('/proxy', proxy('www.google.com', {
  reqAsBuffer: true
}));

reqBodyEncoding

Encoding used to decode request body. Defaults to utf-8.

Use null to preserve as Buffer when proxied request body is a Buffer. (e.g image upload) Accept any values supported by raw-body.

The same encoding is used in the intercept method.

Ignored if parseReqBody is set to false.

app.use('/post', proxy('httpbin.org', {
  reqBodyEncoding: null
}));

timeout

By default, node does not express a timeout on connections.
Use timeout option to impose a specific timeout.
Timed-out requests will respond with 504 status code and a X-Timeout-Reason header.

app.use('/', proxy('httpbin.org', {
  timeout: 2000  // in milliseconds, two seconds
}));

Trace debugging

The node-debug module is used to provide a trace debugging capability.

DEBUG=express-http-proxy-async npm run YOUR_PROGRAM
DEBUG=express-http-proxy-async npm run YOUR_PROGRAM  | grep 'express-http-proxy-async'   # to filter down to just these messages

Will trace the execution of the express-http-proxy-async module in order to aide debugging.

Upgrade to 1.0, transition guide and breaking changes

decorateRequest has been REMOVED, and will generate an error when called. See proxyReqOptDecorator and proxyReqBodyDecorator.

Resolution: Most authors will simply need to change the method name for their decorateRequest method; if author was decorating reqOpts and reqBody in the same method, this will need to be split up.

intercept has been REMOVED, and will generate an error when called. See userResDecorator.

Resolution: Most authors will simply need to change the method name from intercept to userResDecorator, and exit the method by returning the value, rather than passing it to a callback. E.g.:

Before:

app.use('/proxy', proxy('www.google.com', {
  intercept: function(proxyRes, proxyResData, userReq, userRes, cb) {
    data = JSON.parse(proxyResData.toString('utf8'));
    data.newProperty = 'exciting data';
    cb(null,  JSON.stringify(data));
  }
}));

Now:

app.use('/proxy', proxy('www.google.com', {
  userResDecorator: function(proxyRes, proxyResData, userReq, userRes) {
    data = JSON.parse(proxyResData.toString('utf8'));
    data.newProperty = 'exciting data';
    return JSON.stringify(data);
  }
}));

forwardPath and forwardPathAsync have been DEPRECATED and will generate a warning when called. See proxyReqPathResolver.

Resolution: Simple update the name of either forwardPath or forwardPathAsync to proxyReqPathResolver.

When errors occur on your proxy server

When your proxy server responds with an error, express-http-proxy-async returns a response with the same status code. See test/catchingErrors for syntax details.

When your proxy server times out, express-http-proxy-async will continue to wait indefinitely for a response, unless you define a timeout as described above.

Questions

Q: Does it support https proxy?

The library will automatically use https if the provided path has 'https://' or ':443'. You may also set option https to true to always use https.

You can use proxyReqOptDecorator to ammend any auth or challenge headers required to succeed https.

Q: How can I support non-standard certificate chains?

You can use the ability to decorate the proxy request prior to sending. See proxyReqOptDecorator for more details.

app.use('/', proxy('internalhost.example.com', {
  proxyReqOptDecorator: function(proxyReqOpts, originalReq) {
    proxyReqOpts.ca =  [caCert, intermediaryCert]
    return proxyReqOpts;
  }
}))

Release Notes

Release	Notes
1.0.7	Update dependencies. Improve docs on promise rejection. Fix promise rejection on body limit. Improve debug output.
1.0.6	Fixes preserveHostHdr not working, skip userResDecorator on 304, add maybeSkipToNext, test improvements and cleanup.
1.0.5	Minor documentation and test patches
1.0.4	Minor documentation, test, and package fixes
1.0.3	Fixes 'limit option is not taken into account
1.0.2	Minor docs corrections.
1.0.1	Minor docs adjustments.
1.0.0	Major revision. REMOVE decorateRequest, ADD proxyReqOptDecorator and proxyReqBodyDecorator. REMOVE intercept, ADD userResDecorator userResDecorator supports a Promise form for async operations. General cleanup of structure and application of hooks. Documentation improvements. Update all dependencies. Re-organize code as a series of workflow steps, each (potentially) supporting a promise, and creating a reusable pattern for future development.
0.11.0	Allow author to prevent host from being memoized between requests. General program cleanup.
0.10.1	Fixed issue where 'body encoding' was being incorrectly set to the character encoding. Dropped explicit support for node 0.10. Intercept can now deal with gziped responses. Author can now 'force https', even if the original request is over http. Do not call next after ECONNRESET catch.
0.10.0	Fix regression in forwardPath implementation.
0.9.1	Documentation updates. Set 'Accept-Encoding' header to match bodyEncoding.
0.9.0	Better handling for request body when body is JSON.
0.8.0	Features: add forwardPathAsync option Updates: modernize dependencies Fixes: Exceptions parsing proxied response causes error: Can't set headers after they are sent. (#111) If client request aborts, proxied request is aborted too (#107)
0.7.4	Move jscs to devDependencies to avoid conflict with nsp.
0.7.3	Adds a timeout option. Code organization and small bug fixes.
0.7.2	Collecting many minor documentation and test improvements.
0.4.0	Signature of intercept callback changed from function(data, req, res, callback) to function(rsp, data, req, res, callback) where rsp is the original response from the target

Licence

MIT