concat-stream

What is concat-stream?

The concat-stream npm package is a writable stream that concatenates data and calls a callback with the result. It can be used to collect stream data and concatenate it into a single buffer, string, or array. It is useful for collecting stream data from sources like HTTP requests, file reads, and other stream sources.

What are concat-stream's main functionalities?

Concatenate stream data into a buffer

This code sample demonstrates how to use concat-stream to read data from a file and concatenate it into a buffer. The buffer is then converted to a string and logged to the console.

const concat = require('concat-stream');
const fs = require('fs');

const readStream = fs.createReadStream('file.txt');
const concatStream = concat(function(buffer) {
  console.log(buffer.toString());
});

readStream.pipe(concatStream);

Concatenate stream data into a string

This code sample shows how to use concat-stream to collect HTTP response data and concatenate it into a string. The string is then logged to the console.

const concat = require('concat-stream');
const http = require('http');

http.get('http://example.com', function(response) {
  response.pipe(concat(function(data) {
    console.log(data.toString());
  }));
});

Concatenate stream data into an array

This example demonstrates how to use concat-stream to collect data from a readable stream and concatenate it into an array. The resulting array is then logged to the console.

const concat = require('concat-stream');
const { Readable } = require('stream');

const arrayStream = Readable.from(['hello', ' ', 'world']);
const concatStream = concat({ encoding: 'array' }, function(array) {
  console.log(array);
});

arrayStream.pipe(concatStream);

Other packages similar to concat-stream

bl

The 'bl' (Buffer List) package is similar to concat-stream as it provides a storage object for collections of Node.js Buffers. It can be used to collect buffers and access them with a standard readable Buffer interface. It differs in its API and additional features like the ability to access data at specific indices without creating a new Buffer.

stream-buffers

The 'stream-buffers' package offers writable and readable stream implementations using a growable buffer. It is similar to concat-stream in that it allows for the collection of stream data, but it also provides more control over the buffer size and growth.

README

concat-stream

Writable stream that concatenates all the data from a stream and calls a callback with the result. Use this when you want to collect all the data from a stream into a single buffer.

description

Streams emit many buffers. If you want to collect all of the buffers, and when the stream ends concatenate all of the buffers together and receive a single buffer then this is the module for you.

Only use this if you know you can fit all of the output of your stream into a single Buffer (e.g. in RAM).

There are also objectMode streams that emit things other than Buffers, and you can concatenate these too. See below for details.

Related

concat-stream is part of the mississippi stream utility collection which includes more useful stream modules similar to this one.

examples

Buffers

var fs = require('fs')
var concat = require('concat-stream')

var readStream = fs.createReadStream('cat.png')
var concatStream = concat(gotPicture)

readStream.on('error', handleError)
readStream.pipe(concatStream)

function gotPicture(imageBuffer) {
  // imageBuffer is all of `cat.png` as a node.js Buffer
}

function handleError(err) {
  // handle your error appropriately here, e.g.:
  console.error(err) // print the error to STDERR
  process.exit(1) // exit program with non-zero exit code
}

Arrays

var write = concat(function(data) {})
write.write([1,2,3])
write.write([4,5,6])
write.end()
// data will be [1,2,3,4,5,6] in the above callback

Uint8Arrays

var write = concat(function(data) {})
var a = new Uint8Array(3)
a[0] = 97; a[1] = 98; a[2] = 99
write.write(a)
write.write('!')
write.end(Buffer.from('!!1'))

See test/ for more examples

methods

var concat = require('concat-stream')

var writable = concat(opts={}, cb)

Return a writable stream that will fire cb(data) with all of the data that was written to the stream. Data can be written to writable as strings, Buffers, arrays of byte integers, and Uint8Arrays.

By default concat-stream will give you back the same data type as the type of the first buffer written to the stream. Use opts.encoding to set what format data should be returned as, e.g. if you if you don't want to rely on the built-in type checking or for some other reason.

• string - get a string
• buffer - get back a Buffer
• array - get an array of byte integers
• uint8array, u8, uint8 - get back a Uint8Array
• object, get back an array of Objects

If you don't specify an encoding, and the types can't be inferred (e.g. you write things that aren't in the list above), it will try to convert concat them into a Buffer.

If nothing is written to writable then data will be an empty array [].

error handling

concat-stream does not handle errors for you, so you must handle errors on whatever streams you pipe into concat-stream. This is a general rule when programming with node.js streams: always handle errors on each and every stream. Since concat-stream is not itself a stream it does not emit errors.

We recommend using end-of-stream or pump for writing error tolerant stream code.

license

MIT LICENSE