fatbuffer

Node FatBuffer

NodeJS Buffer interface has a maximum length of 2147483647 bytes, but what if we need more ?

One solution is to distribute the data on multiple buffers and map each index to one of the buffers. This is what the library is about, no need to do the mapping, you'll get the same interface as the original 'Buffer' but with infinite Buffer size which is around (2147483647 * 2^32 -1) = 9 x 10^9 gb).

Limitations:

not all Buffer methods are supported.

Installation

npm install fatbuffer --save

Initializing a FatBuffer

FatBuffer.alloc(length)

Methods:

.isBuffer(obj)

returns true if obj was a Buffer or an instance of FatBuffer.

.isFatBuffer(obj)

returns  true if obj was an instance of FatBuffer.

Parameters:

length: length in bytes.

Properties

.length

Returns length in bits.

Usage

const FatBuffer = require("fatbuffer")

const buffer = FatBuffer.alloc(2147483647*5);

buffer[2147483647*3]=255;
console.log(buffer[2147483647*3]) // 255

Implementation

Array of buffers and some mapping techniques.