pbf

pbf

A low-level, fast, ultra-lightweight (3KB gzipped) JavaScript library for decoding and encoding protocol buffers, a compact binary format for structured data serialization. Works both in Node and the browser. Supports lazy decoding and detailed customization of the reading/writing code.

Performance

This library is fast — competitive with or faster than other JS protobuf implementations, and orders of magnitude smaller. Here's a result from a real-world benchmark on Node v26 (decoding and encoding 439 Mapbox vector tiles, 37.5 MB total; the equivalent JSON is 136 MB):

	decode	encode
pbf	200ms, 187 MB/s	188ms, 200 MB/s
protocol-buffers	297ms, 126 MB/s	620ms, 60 MB/s
protobuf.js	226ms, 169 MB/s	510ms, 74 MB/s
JSON	488ms, 278 MB/s	267ms, 509 MB/s

JSON throughput is measured against the 136 MB JSON payload, not the 37.5 MB pbf payload — on the same data, pbf is ~2× faster to decode and ~2.5× faster to encode, and produces output roughly a quarter the size. See bench/bench-tiles.js.

Examples

Using Compiled Code

Install pbf and compile a JavaScript module from a .proto file:

$ npm install -g pbf
$ pbf example.proto > example.js

Then read and write objects using the module like this:

import {PbfReader, PbfWriter} from 'pbf';
import {readExample, writeExample} from './example.js';

// read
const obj = readExample(new PbfReader(buffer));

// write
const pbf = new PbfWriter();
writeExample(obj, pbf);
const buffer = pbf.finish();

Alternatively, you can compile a protobuf schema file directly in the code:

import {compile} from 'pbf/compile';
import schema from 'protocol-buffers-schema';

const proto = schema.parse(fs.readFileSync('example.proto'));
const {readExample, writeExample} = compile(proto);

Custom Reading

const data = new PbfReader(buffer).readFields(readData, {});

function readData(tag, data, pbf) {
    if (tag === 1) data.name = pbf.readString();
    else if (tag === 2) data.version = pbf.readVarint();
    else if (tag === 3) data.layer = pbf.readMessage(readLayer, {});
}
function readLayer(tag, layer, pbf) {
    if (tag === 1) layer.name = pbf.readString();
    else if (tag === 3) layer.size = pbf.readVarint();
}

Custom Writing

const pbf = new PbfWriter();
writeData(data, pbf);
const buffer = pbf.finish();

function writeData(data, pbf) {
    pbf.writeStringField(1, data.name);
    pbf.writeVarintField(2, data.version);
    pbf.writeMessage(3, writeLayer, data.layer);
}
function writeLayer(layer, pbf) {
    pbf.writeStringField(1, layer.name);
    pbf.writeVarintField(2, layer.size);
}

Install

Install using NPM with npm install pbf, then import as a module:

import {PbfReader, PbfWriter} from 'pbf';

Or use as a module directly in the browser with jsDelivr:

<script type="module">
    import {PbfReader, PbfWriter} from 'https://cdn.jsdelivr.net/npm/pbf/+esm';
</script>

Alternatively, there's a browser bundle exposing a Pbf global with PbfReader and PbfWriter properties:

<script src="https://cdn.jsdelivr.net/npm/pbf"></script>

API

The library exposes two classes: PbfReader for decoding and PbfWriter for encoding. Splitting them lets bundlers tree-shake the half you don't use.

Create a PbfReader from a Buffer or Uint8Array:

// parse a pbf file from disk in Node
const pbf = new PbfReader(fs.readFileSync('data.pbf'));

// parse a pbf file in a browser after an ajax request with responseType="arraybuffer"
const pbf = new PbfReader(new Uint8Array(xhr.response));

Both classes expose the following properties:

pbf.length; // length of the underlying buffer
pbf.pos; // current offset for reading or writing

Reading

Read a sequence of fields:

pbf.readFields((tag) => {
    if (tag === 1) pbf.readVarint();
    else if (tag === 2) pbf.readString();
    else ...
});

It optionally accepts an object that will be passed to the reading function for easier construction of decoded data, and also passes the PbfReader object as a third argument:

const result = pbf.readFields(readField, {})

function readField(tag, result, pbf) {
    if (tag === 1) result.id = pbf.readVarint();
}

To read an embedded message, use pbf.readMessage(fn[, obj]) (in the same way as read).

Read values:

const value = pbf.readVarint();
const str = pbf.readString();
const numbers = pbf.readPackedVarint();

For lazy or partial decoding, simply save the position instead of reading a value, then later set it back to the saved value and read:

const fooPos = -1;
pbf.readFields((tag) => {
    if (tag === 1) fooPos = pbf.pos;
});
...
pbf.pos = fooPos;
pbf.readMessage(readFoo);

Scalar reading methods:

• readVarint(isSigned) (pass true if you expect negative varints)
• readSVarint()
• readFixed32()
• readFixed64()
• readSFixed32()
• readSFixed64()
• readBoolean()
• readFloat()
• readDouble()
• readString()
• readBytes()
• skip(value)

Packed reading methods:

• readPackedVarint(arr, isSigned) (appends read items to arr)
• readPackedSVarint(arr)
• readPackedFixed32(arr)
• readPackedFixed64(arr)
• readPackedSFixed32(arr)
• readPackedSFixed64(arr)
• readPackedBoolean(arr)
• readPackedFloat(arr)
• readPackedDouble(arr)

Writing

Create a PbfWriter (optionally with a pre-allocated Buffer or Uint8Array):

const pbf = new PbfWriter();

Write values:

pbf.writeVarint(123);
pbf.writeString("Hello world");

Write an embedded message:

pbf.writeMessage(1, writeObj, obj);

function writeObj(obj, pbf) {
    pbf.writeStringField(obj.name);
    pbf.writeVarintField(obj.version);
}

Field writing methods:

• writeVarintField(tag, val)
• writeSVarintField(tag, val)
• writeFixed32Field(tag, val)
• writeFixed64Field(tag, val)
• writeSFixed32Field(tag, val)
• writeSFixed64Field(tag, val)
• writeBooleanField(tag, val)
• writeFloatField(tag, val)
• writeDoubleField(tag, val)
• writeStringField(tag, val)
• writeBytesField(tag, buffer)

Packed field writing methods:

• writePackedVarint(tag, val)
• writePackedSVarint(tag, val)
• writePackedSFixed32(tag, val)
• writePackedSFixed64(tag, val)
• writePackedBoolean(tag, val)
• writePackedFloat(tag, val)
• writePackedDouble(tag, val)

Scalar writing methods:

• writeVarint(val)
• writeSVarint(val)
• writeSFixed32(val)
• writeSFixed64(val)
• writeBoolean(val)
• writeFloat(val)
• writeDouble(val)
• writeString(val)
• writeBytes(buffer)

Message writing methods:

• writeMessage(tag, fn[, obj])
• writeRawMessage(fn[, obj])

Misc methods:

• realloc(minBytes) - pad the underlying buffer size to accommodate the given number of bytes; note that the size increases exponentially, so it won't necessarily equal the size of data written
• finish() - make the current buffer ready for reading and return the data as a buffer slice

For an example of a real-world usage of the library, see vector-tile-js.

Proto Schema to JavaScript

If installed globally, pbf provides a binary that compiles proto files into JavaScript modules. Usage:

$ pbf <proto_path> [--no-write] [--no-read] [--legacy]

The --no-write and --no-read switches remove corresponding code in the output. The --legacy switch makes it generate a CommonJS module instead of ESM.

Pbf will generate read<Identifier> and write<Identifier> functions for every message in the schema. For nested messages, their names will be concatenated — e.g. Message inside Test will produce readTestMessage and writeTestMessage functions.

• read(pbf) - decodes an object from the given PbfReader instance.
• write(obj, pbf) - encodes an object into the given PbfWriter instance (usually empty).

The resulting code is clean and simple, so it's meant to be customized.

Similar packages

Other packages similar to pbf

protobufjs

protobufjs is a more feature-rich library for working with Protocol Buffers in JavaScript. Unlike pbf, which focuses on minimalism and speed, protobufjs offers a comprehensive suite of tools including a CLI for generating static code and a more extensive API for manipulating protobuf schemas dynamically. This makes protobufjs better suited for applications requiring complex protobuf handling.

google-protobuf

google-protobuf is the official Protocol Buffers library for JavaScript, developed by Google. It provides robust support for the full range of protobuf features, including reflection and extensions, which are not supported by pbf. This package is ideal for users looking for official support and the most complete implementation of the Protocol Buffers specification.