pumpify

What is pumpify?

The pumpify npm package is a module that combines an array of streams into a single duplex stream using pump and duplexify. It handles the piping between streams for you, and destroys all streams if one of them closes. This is particularly useful when you want to work with multiple streams in a sequential manner, ensuring that data is properly piped through each stage and that resources are cleaned up if an error occurs.

What are pumpify's main functionalities?

Combining streams into a single duplex stream

This code sample demonstrates how to create a single duplex stream that reads from a file, compresses the data using gzip, and then writes the compressed data to a new file. The pumpify module handles the piping and stream destruction.

const pumpify = require('pumpify');
const fs = require('fs');
const zlib = require('zlib');
const stream = pumpify(fs.createReadStream('input.txt'), zlib.createGzip(), fs.createWriteStream('output.txt.gz'));

Error handling across piped streams

This code sample shows how to handle errors in a pipeline created with pumpify. If any of the streams in the pipeline emit an error, pumpify will destroy all streams and emit the error on the resulting duplex stream.

const pumpify = require('pumpify');
const fs = require('fs');
const zlib = require('zlib');
const stream = pumpify(fs.createReadStream('input.txt'), zlib.createGzip(), fs.createWriteStream('output.txt.gz'));
stream.on('error', err => console.error('Stream error:', err));

Other packages similar to pumpify

multistream

Multistream is a module that allows you to create a single readable stream from multiple streams. It is similar to pumpify in that it deals with multiple streams, but it does not create a duplex stream and does not handle the piping and destruction of streams automatically.

stream-combiner2

Stream-combiner2 is another module that combines multiple streams into one. It is similar to pumpify but uses a different implementation. It also turns a pipeline of streams into a single duplex stream, but it does not use the pump module for error handling and stream destruction.

duplexify

Duplexify is a module that turns a writable and readable stream into a single duplex stream. While pumpify uses duplexify under the hood, duplexify itself does not handle the combination of multiple streams or the automatic destruction of streams on error.

README

pumpify

Combine an array of streams into a single duplex stream using pump and duplexify. If one of the streams closes/errors all streams in the pipeline will be destroyed.

npm install pumpify

Usage

Pass the streams you want to pipe together to pumpify pipeline = pumpify(s1, s2, s3, ...). pipeline is a duplex stream that writes to the first streams and reads from the last one. Streams are piped together using pump so if one of them closes all streams will be destroyed.

var pumpify = require('pumpify')
var tar = require('tar-fs')
var zlib = require('zlib')
var fs = require('fs')

var untar = pumpify(zlib.createGunzip(), tar.extract('output-folder'))
// you can also pass an array instead
// var untar = pumpify([zlib.createGunzip(), tar.extract('output-folder')])

fs.createReadStream('some-gzipped-tarball.tgz').pipe(untar)

If you are pumping object streams together use pipeline = pumpify.obj(s1, s2, ...). Call pipeline.destroy() to destroy the pipeline (including the streams passed to pumpify).

Using setPipeline(s1, s2, ...)

Similar to duplexify you can also define the pipeline asynchronously using setPipeline(s1, s2, ...)

var untar = pumpify()

setTimeout(function() {
  // will start draining the input now
  untar.setPipeline(zlib.createGunzip(), tar.extract('output-folder'))
}, 1000)

fs.createReadStream('some-gzipped-tarball.tgz').pipe(untar)

License

MIT

Related

pumpify is part of the mississippi stream utility collection which includes more useful stream modules similar to this one.