multipipe

What is multipipe?

The 'multipipe' npm package is a utility for creating a pipeline of streams in Node.js. It allows you to easily combine multiple streams into a single pipeline, handling errors and ensuring proper cleanup of resources.

What are multipipe's main functionalities?

Combining Multiple Streams

This feature allows you to combine multiple streams into a single pipeline. In this example, a file is read, decompressed using gzip, and then extracted using tar-stream. The 'multipipe' function handles the piping and error management.

const multipipe = require('multipipe');
const fs = require('fs');
const zlib = require('zlib');
const tar = require('tar-stream');

const extract = tar.extract();
extract.on('entry', (header, stream, next) => {
  stream.on('end', next);
  stream.resume();
});

const pipeline = multipipe(
  fs.createReadStream('archive.tar.gz'),
  zlib.createGunzip(),
  extract
);

pipeline.on('error', (err) => {
  console.error('Pipeline failed:', err);
});

pipeline.on('finish', () => {
  console.log('Pipeline succeeded');
});

Error Handling

This feature demonstrates how 'multipipe' handles errors in the pipeline. If any stream in the pipeline encounters an error, the 'error' event is emitted, and you can handle it appropriately. In this example, an error will occur because the input file does not exist.

const multipipe = require('multipipe');
const fs = require('fs');
const zlib = require('zlib');

const pipeline = multipipe(
  fs.createReadStream('nonexistentfile.gz'),
  zlib.createGunzip(),
  fs.createWriteStream('output.txt')
);

pipeline.on('error', (err) => {
  console.error('Pipeline error:', err);
});

Other packages similar to multipipe

pump

'pump' is a similar package that also helps in piping streams together and managing errors. It is simpler and more lightweight compared to 'multipipe', but it does not offer the same level of flexibility in terms of combining multiple streams.

stream-combiner2

'stream-combiner2' is another package that allows you to combine multiple streams into one. It is similar to 'multipipe' but focuses more on simplicity and ease of use. It also handles errors and ensures proper cleanup of resources.

README

multipipe

A better Stream#pipe that creates duplex streams and lets you handle errors in one place. With promise support!

Example

const pipe = require('multipipe')

// pipe streams
const stream = pipe(streamA, streamB, streamC)

// centralized error handling
stream.on('error', fn)

// creates a new stream
source.pipe(stream).pipe(dest)

// optional callback on finish or error
pipe(streamA, streamB, streamC, err => {
  // ...
})

// pass options
pipe(streamA, streamB, streamC, {
  objectMode: false
})

// await finish
await pipe(streamA, streamB, streamC)

Duplex streams

Write to the pipe and you'll really write to the first stream, read from the pipe and you'll read from the last stream.

const stream = pipe(a, b, c)

source
  .pipe(stream)
  .pipe(destination)

In this example the flow of data is:

• source ->
• a ->
• b ->
• c ->
• destination

Error handling

Each pipe forwards the errors the streams it wraps emit, so you have one central place to handle errors:

const stream = pipe(a, b, c)

stream.on('error', err => {
  // called three times
})

a.emit('error', new Error)
b.emit('error', new Error)
c.emit('error', new Error)

API

pipe(stream...[, opts][, cb])

Pass a variable number of streams and each will be piped to the next one.

A stream will be returned that wraps passed in streams in a way that errors will be forwarded and you can write to and/or read from it.

The returned stream is also a Promise that will resolve on finish and reject on error.

Pass an object as the second to last or last argument to pass as options to the underlying stream constructors.

Pass a function as last argument to be called on error or finish of the last stream.

pipe(streams[, cb])

You can also pass an Array of streams if that is more convenient.

Installation

$ npm install multipipe

License

The MIT License (MIT)

Copyright (c) 2014 Segment.io Inc. <[e-mail]>

Copyright (c) 2014 Julian Gruber <[e-mail]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.