inflate-auto

InflateAuto

The InflateAuto class is designed to function as a drop-in replacement for zlib.Gunzip, zlib.Inflate, and zlib.InflateRaw when the method of compression is not known in advance. InflateAuto uses the first few bytes of data to determine the compression method (by checking for a valid gzip or zlib deflate header) then delegating the decompression work to the corresponding type.

Introductory Example

const InflateAuto = require('inflate-auto');
const assert = require('assert');
const zlib = require('zlib');

const testData = new Buffer('example data');
const compressor = Math.random() < 0.33 ? zlib.deflate :
      Math.random() < 0.5 ? zlib.deflateRaw :
      zlib.gzip;
compressor(testData, (errCompress, compressed) => {
  assert.ifError(errCompress);

  InflateAuto.inflateAuto(compressed, function(errDecompress, decompressed) {
    assert.ifError(errDecompress);
    assert.deepStrictEqual(decompressed, testData);
    console.log('Data compressed with random format and auto-decompressed.');
  });
});

Compatibility

InflateAuto should behave identically to any of the zlib decompression types, with the exception of instanceof and .constructor checks. Using the class should be as simple as s/Inflate(Raw)?/InflateAuto/g in existing code. If any real-world code requires modification (other than mentioned above) to work with InflateAuto it is considered a bug in InflateAuto. Please report any such issues.

Installation

This package can be installed using npm by running:

npm install inflate-auto

Recipes

Deflate HTTP

The primary use case for which this module was created is decompressing HTTP responses which declare Content-Encoding: deflate. As noted in Section 4.2.2 of RFC 7230 "Some non-conformant implementations send the "deflate" compressed data without the zlib wrapper." This has been attributed to early Microsoft servers and to old Apache mod_deflate, and is an issue in several less common servers. Regardless of the most common cause, it is observed in real-world behavior and poses a compatibility risk for HTTP clients which support deflate encoding. Using InflateAuto is one way to address the issue.

Compressed HTTP/HTTPS responses can be supported with code similar to the following:

const InflateAuto = require('inflate-auto');
const https = require('https');
const url = require('url');
const zlib = require('zlib');

const options = url.parse('https://api.stackexchange.com/2.2/answers?order=desc&sort=activity&site=stackoverflow');
options.headers = {
  Accept: 'application/json',
  'Accept-Encoding': 'gzip, deflate'
};
https.get(options, function(res) {
  const encoding =
    (res.headers['content-encoding'] || 'identity').trim().toLowerCase();

  // InflateAuto could be used for gzip to accept deflate data declared as gzip
  const inflater = encoding === 'deflate' ? new InflateAuto() :
    encoding === 'gzip' ? new zlib.Gunzip() :
    null;

  res.on('error', err => console.error('Response error:', err));

  let bodyData;
  if (inflater) {
    inflater.on('error', err => console.error('Decompression error:', err));
    bodyData = res.pipe(inflater);
  } else {
    bodyData = res;
  }

  bodyData.pipe(process.stdout, {end: false});
})
  .on('error', err => console.error('Request error:', err));

Log Compression Format

To be notified when the compression format is determined, listen for the 'format' event as follows:

const InflateAuto = require('inflate-auto');
const inflater = new InflateAuto();
inflater.on(
  'format',
  decoder => console.log('Compression format: ' + decoder.constructor.name)
);
inflater.write(compressedData);

Inflate Possibly-Compressed Data

By specifying PassThrough as the default format, InflateAuto can be used to inflate compressed data and pass through other data unchanged as follows:

const InflateAuto = require('inflate-auto');
const stream = require('stream');
const inflater = new InflateAuto({defaultFormat: stream.PassThrough});
inflater.pipe(process.stdout);
inflater.end(compressedOrUncompressedData);

Note that the above code would treat "raw" DEFLATE data as uncompressed since InflateRaw is normally the default format and is overridden with PassThrough. Feel free to open an issue to request support for detecting "raw" DEFLATE if this is desired.

Synchronous Inflate

Data can be decompressed while blocking the main thread using InflateAuto.inflateAutoSync (analogously to zlib.inflateSync) as follows:

const InflateAuto = require('inflate-auto');
const assert = require('assert');
const zlib = require('zlib');

const compressor = Math.random() < 0.33 ? zlib.deflateSync :
      Math.random() < 0.5 ? zlib.deflateRawSync :
      zlib.gzipSync;
const testData = new Buffer('example data');
const compressed = compressor(testData);
const decompressed = InflateAuto.inflateAutoSync(compressed);
assert.deepStrictEqual(decompressed, testData);

More examples can be found in the test specifications.

API Docs

For the details of using this module as a library, see the API Documentation.

Contributing

Contributions are appreciated. Contributors agree to abide by the Contributor Covenant Code of Conduct. If this is your first time contributing to a Free and Open Source Software project, consider reading How to Contribute to Open Source in the Open Source Guides.

If the desired change is large, complex, backwards-incompatible, can have significantly differing implementations, or may not be in scope for this project, opening an issue before writing the code can avoid frustration and save a lot of time and effort.

License

This project is available under the terms of the MIT License. See the summary at TLDRLegal.