bl

What is bl?

The 'bl' (Buffer List) npm package is a utility that provides a storage mechanism for Node.js Buffer data. It allows for the collection of Buffer objects, and provides a way to access the combined data as a single contiguous Buffer or a stream. It is useful for handling streams of binary data and can simplify the process of collecting and manipulating this data.

What are bl's main functionalities?

Collecting stream data

This code sample demonstrates how to collect data from an HTTP response stream and convert it to a string once the stream ends.

const { BufferList } = require('bl');
const bl = new BufferList();
require('http').get('http://example.com', (res) => {
  res.pipe(bl);
  res.on('end', () => {
    console.log(bl.toString());
  });
});

Appending buffers and strings

This code sample shows how to append both strings and Buffer objects to a BufferList instance, and then convert the entire list to a string.

const { BufferList } = require('bl');
const bl = new BufferList();
bl.append('first string ');
bl.append(Buffer.from('second string'));
console.log(bl.toString());

Random access to data

This code sample illustrates how to perform random access on the data within a BufferList by using the slice method to retrieve a portion of the data.

const { BufferList } = require('bl');
const bl = new BufferList();
bl.append('hello ');
bl.append('world');
console.log(bl.slice(0, 5).toString()); // 'hello'

Duplex stream compatibility

This code sample demonstrates how BufferList can be used as a duplex stream, where data piped into it can be manipulated and then piped out to another destination.

const { BufferListStream } = require('bl');
const blStream = new BufferListStream();
process.stdin.pipe(blStream).pipe(process.stdout);

Other packages similar to bl

concat-stream

The 'concat-stream' package is similar to 'bl' in that it collects stream data into buffers and then concatenates them. It differs in its API and the way it handles the output, providing a callback function to access the concatenated result.

buffers

The 'buffers' package provides a way to manage a collection of Node.js Buffer objects, similar to 'bl'. It offers a different API for buffer manipulation, including methods for slicing and dicing buffer collections.

bufferstreams

The 'bufferstreams' package is another alternative that allows for buffering of streaming data into a single Buffer or string. It is similar to 'bl' but focuses more on providing a stream interface for buffering and less on direct buffer manipulation.

README

bl (BufferList)

A Node.js Buffer list collector, reader and streamer thingy.

bl is a storage object for collections of Node Buffers, exposing them with the main Buffer readable API. Also works as a duplex stream so you can collect buffers from a stream that emits them and emit buffers to a stream that consumes them!

The original buffers are kept intact and copies are only done as necessary. Any reads that require the use of a single original buffer will return a slice of that buffer only (which references the same memory as the original buffer). Reads that span buffers perform concatenation as required and return the results transparently.

const BufferList = require('bl')

var bl = new BufferList()
bl.append(new Buffer('abcd'))
bl.append(new Buffer('efg'))
bl.append(new Buffer('hi'))
bl.append(new Buffer('j'))
bl.append(new Buffer([ 0x3, 0x4 ])

console.log(bl.length) // 12

console.log(bl.slice(0, 10).toString('ascii')) // 'abcdefghij'
console.log(bl.slice(3, 10).toString('ascii')) // 'defghij'
console.log(bl.slice(3, 6).toString('ascii'))  // 'def'
console.log(bl.slice(3, 8).toString('ascii'))  // 'defgh'
console.log(bl.slice(5, 10).toString('ascii')) // 'fghij'

console.log(bl.readUInt16BE(10)) // 0x0304
console.log(bl.readUInt16LE(10)) // 0x0403

Give it a callback in the constructor and use it just like concat-stream:

const BufferList = require('bl')
    , fs         = require('fs')

var bl = new BufferList(function (data) {
	console.log(data.toString())
})

fs.createReadStream('README.md').pipe(bl)

Or, use it as a readable stream:

const BufferList = require('bl')
    , fs         = require('fs')

var bl = new BufferList()
bl.append(new Buffer('abcd'))
bl.append(new Buffer('efg'))
bl.append(new Buffer('hi'))
bl.append(new Buffer('j'))

bl.pipe(fs.createWriteStream('gibberish.txt'))

API

• new BufferList([ callback ])
• bl.length
• bl.append(buffer)
• bl.get(index)
• bl.slice([ start ], [ end ])
• bl.consume(bytes)
• bl.readDoubleBE(), bl.readDoubleLE(), bl.readFloatBE(), bl.readFloatLE(), bl.readInt32BE(), bl.readInt32LE(), bl.readUInt32BE(), bl.readUInt32LE(), bl.readInt16BE(), bl.readInt16LE(), bl.readUInt16BE(), bl.readUInt16LE(), bl.readInt8(), bl.readUInt8()
• Streams

---

new BufferList([ callback ])

The constructor takes an optional callback, if supplied, the callback will be called with the total contents of the list, concatenated together (bl.slice()) when bl.end() is called (from a piped stream).

Normally, no arguments are required for the constructor.

---

bl.length

Get the length of the list in bytes. This is the sum of the lengths of all of the buffers contained in the list, minus any initial offset for a semi-consumed buffer at the beginning. Should accurately represent the total number of bytes that can be read from the list.

---

bl.append(buffer)

append(buffer) adds an additional buffer to the internal list.

---

bl.get(index)

get() will return the byte at the specified index.

---

bl.slice([ start, [ end ] ])

slice() returns a new Buffer object containing the bytes within the range specified. Both start and end are optional and will default to the beginning and end of the list respectively.

If the requested range spans a single internal buffer then a slice of that buffer will be returned which shares the original memory range of that Buffer. If the range spans multiple buffers then copy operations will likely occur to give you a uniform Buffer.

---

bl.consume(bytes)

consume() will shift bytes off the start of the list. The number of bytes consumed don't need to line up with the sizes of the internal Buffers—initial offsets will be calculated accordingly in order to give you a consistent view of the data.

---

bl.readDoubleBE(), bl.readDoubleLE(), bl.readFloatBE(), bl.readFloatLE(), bl.readInt32BE(), bl.readInt32LE(), bl.readUInt32BE(), bl.readUInt32LE(), bl.readInt16BE(), bl.readInt16LE(), bl.readUInt16BE(), bl.readUInt16LE(), bl.readInt8(), bl.readUInt8()

All of the standard byte-reading methods of the Buffer interface are implemented and will operate across internal Buffer boundaries transparently.

See the Buffer documentation for how these work.

---

Streams

bl is a Node Duplex Stream, so it can be read from and written to like a standard Node stream. You can also pipe() to and from a bl instance.

License

bl is Copyright (c) 2013 Rod Vagg @rvagg and licenced under the MIT licence. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE file for more details.