What is protobufjs?
The protobufjs npm package provides a comprehensive suite of tools for working with Protocol Buffers (protobuf), a method of serializing structured data. It allows users to encode and decode protobuf messages, generate and work with static code, and handle dynamic message building and parsing.
What are protobufjs's main functionalities?
Loading .proto files
This feature allows users to load .proto files and use the defined protobuf structures within their JavaScript code.
const protobuf = require('protobufjs');
protobuf.load('awesome.proto', function(err, root) {
if (err) throw err;
const AwesomeMessage = root.lookupType('awesomepackage.AwesomeMessage');
// ... use AwesomeMessage
});
Encoding and decoding messages
With protobufjs, users can encode JavaScript objects into binary protobuf format and decode binary messages into JavaScript objects.
const message = AwesomeMessage.create({ awesomeField: 'AwesomeString' });
const buffer = AwesomeMessage.encode(message).finish();
const decodedMessage = AwesomeMessage.decode(buffer);
Reflection and runtime message building
This feature allows users to work with protobuf messages dynamically at runtime using JSON descriptors, without the need for generated static code.
const root = protobuf.Root.fromJSON(jsonDescriptor);
const AwesomeMessage = root.lookupType('awesomepackage.AwesomeMessage');
const errMsg = AwesomeMessage.verify({ awesomeField: 'AwesomeString' });
if (errMsg) throw Error(errMsg);
const message = AwesomeMessage.create({ awesomeField: 'AwesomeString' });
Static code generation
Protobufjs can generate static code from .proto files, which can be used for better performance and type safety.
protobuf.load('awesome.proto', function(err, root) {
if (err) throw err;
protobuf.codegen(root, { keepCase: true }, function(err, output) {
if (err) throw err;
// output will contain the generated static code
});
});
Other packages similar to protobufjs
@apollo/protobufjs
This is a fork of the original protobufjs package with some modifications. It is used within the Apollo tooling ecosystem but generally offers similar functionality to protobufjs.
google-protobuf
This is the official Protocol Buffers runtime library for JavaScript. It is provided by Google and offers similar serialization and deserialization capabilities. However, it may not be as feature-rich or flexible as protobufjs in terms of dynamic message handling and may require more setup for code generation.
pbf
Pbf is a fast, lightweight Protocol Buffers implementation in JavaScript. It focuses on performance and is smaller in size compared to protobufjs. However, it might not offer the same level of functionality, especially in terms of reflection and dynamic message building.
A protobuf implementation on top of ByteBuffer.js including a .proto parser,
reflection, message class building and simple encoding and decoding in plain JavaScript. No compilation step required,
works out of the box on .proto files. It's also used for the cross-platform multiplayer component in
eSoccer, an HTML5 game developed at University of Applied Sciences Bonn.
Features
Builder
Probably the core component of ProtoBuf.js. Resolves all type references, performs all the necessary checks and returns
ready to use classes. Can be created from a .proto file or from a JSON definition. The later does not even require the
.proto parser to be included (see).
Install: npm install protobufjs
Using .proto files
Example: tests/complex.proto
var ProtoBuf = require("protobufjs");
var builder = ProtoBuf.protoFromFile("tests/complex.proto");
var Game = builder.build("Game");
var Car = Game.Cars.Car;
var car = new Car("Rusty", new Car.Vendor("Iron Inc.", new Car.Vendor.Address("US")), Car.Speed.SUPERFAST);
var car = new Car({
"model": "Rusty",
"vendor": {
"name": "Iron Inc.",
"address": {
"country": "US"
}
},
"speed": "SUPERFAST"
});
var buffer = car.encode();
var socket = ...;
socket.send(buffer.toArrayBuffer());
socket.send(car.toArrayBuffer());
Using JSON without the .proto parser
Example: tests/complex.json
var ProtoBuf = require("protobufjs");
var builder = ProtoBuf.newBuilder();
builder.define("Game");
builder.create([
{
"name": "Car",
"fields": [
{
"rule": "required",
"type": "string",
"name": "model",
"id": 1
},
...
],
"messages": [
{
"name": "Vendor",
"fields": ...,
},
...
],
"enums": [
{
"name": "Speed",
"values": [
{
"name": "FAST",
"id": 1
},
...
]
}
]
}
]);
var Game = builder.build("Game");
var Car = Game.Cars.Car;
... actually the same as above ...
When using JSON only, you can use ProtoBuf.noparse.js
/ ProtoBuf.noparse.min.js instead, which
do NOT include the ProtoBuf.DotProto
package for parsing and are therefore even smaller.
Command line utility
It's also possible to transform .proto files into their JSON counterparts or to transform entire namespaces into
ready-to-use message classes and enum objects by using the proto2js
command line utility.
Usage: proto2js protoFile [-class[=My.Package]|-commonjs[=My.Package]|-amd[=My.Package]] [-min] [> outFile]
Options:
-class[=My.Package] Creates the classes instead of just a JSON definition.
If you do not specifiy a package, the package
declaration from the .proto file is used instead.
-commonjs[=My.Package] Creates a CommonJS export instead of just a JSON def.
If you do not specifiy a package, the package
declaration from the .proto file is used instead.
-amd[=My.Package] Creates an AMD define instead of just a JSON def.
If you do not specifiy a package, the package
declaration from the .proto file is used instead.
-min Minifies the output.
So, to create a JSON definition from the tests/complex.proto
file, run:
proto2js tests/complex.proto > tests/complex.json
Or to create classes/a CommonJS export/an AMD define for the entire Game
namespace, run:
proto2js tests/complex.proto -class=Game > tests/complex.js
proto2js tests/complex.proto -commonjs=Game > tests/complex.js
proto2js tests/complex.proto -amd=Game.Cars > tests/complex.js
Parser
Only available in the full build (i.e. not in "noparse" builds). Compliant with the protobuf parser to the following extend:
-
Required, optional, repeated and packed repeated fields:
message Test {
required int32 a = 1;
optional int32 b = 2 [default=100];
repeated int32 c = 3;
repeated int32 c = 4 [packed=true];
}
-
Data types: int32, uint32, sint32, bool, enum, string, bytes, messages, embedded messages, fixed32, sfixed32, float, double:
message Test {
required int32 a = 1; // Varint encoded
required uint32 b = 2; // Varint encoded
required sint32 c = 3; // Varint zigzag encoded
required bool d = 4; // Varint encoded
enum Priority {
LOW = 1;
MEDIUM = 2;
HIGH = 3;
}
optional Priority e = 5 [default=MEDIUM]; // Varint encoded
required string f = 6; // Varint length delimited
required bytes g = 7; // Varint length delimited
required Embedded h = 8; // Varint length delimited
message Embedded {
repeated int32 a = 1; // Multiple tags
repeated int32 b = 2 [packed=true]; // One tag, length delimited
required fixed32 c = 3; // Fixed 4 bytes
required sfixed32 d = 4; // Fixed 4 bytes zigzag encoded
required float e = 5; // Fixed 4 bytes
required double f = 6; // Fixed 8 bytes
}
}
-
Packages
package My.Game;
message Test {
...
}
message Test2 {
required My.Game.Test test = 1;
}
-
Qualified and fully qualified name resolving:
package My.Game;
message Test {
...
enum Priority {
LOW = 1;
MEDIUM = 2;
HIGH = 3;
}
}
message Test2 {
required .My.Game.Test.Priority priority_fqn = 1 [default=LOW];
required Test.Priority priority_qn = 2 [default=MEDIUM];
}
-
Options on all levels:
option toplevel_1 = 10;
option toplevel_2 = "Hello!";
message Test {
option inmessage = "World!";
optional int32 somenumber = 1 [default=123]; // Actually the only one used
}
Not (yet) supported
- Extensions (what for?), imports (put everything into one builder instead by calling
Builder#define(package)
and
Builder#create([messages])
multiple times) and services (you roll your own, don't you?).
However, if you need anything of the above, please drop me a note how you'd like to see it implemented. It's just that
I have no idea how to benefit from that and therefore I am not sure how to design it.
Calling the parser on your own
var ProtoBuf = require("protobufjs"),
fs = require("fs"),
util = require("util");
var parser = new ProtoBuf.DotProto.Parser(fs.readFileSync("tests/complex.proto"));
var ast = parser.parse();
console.log(util.inspect(ast, false, null, true));
Encoder / Decoder
Built into all message classes. Just call YourMessage#encode([buffer])
respectively YourMessage.decode(buffer)
.
Encoding a message
...
var YourMessage = builder.build("YourMessage");
var myMessage = new YourMessage(...);
var byteBuffer = myMessage.encode();
var buffer = byteBuffer.toArrayBuffer();
var buffer = myMessage.toArrayBuffer();
var socket = ...;
socket.send(buffer);
Decoding from an ArrayBuffer/ByteBuffer
...
var YourMessage = builder.build("YourMessage");
var buffer = ...;
var myMessage = YourMessage.decode(buffer);
CommonJS, AMD and browser (shim)
node.js / CommonJS
npm install protobufjs
var ProtoBuf = require("protobufjs");
...
RequireJS / AMD
define(["ProtoBuf"], function(ProtoBuf) {
...
});
Browser (shim)
<script src="//raw.github.com/dcodeIO/ProtoBuf.js/master/ProtoBuf.min.js"></script>
var ProtoBuf = dcodeIO.ProtoBuf;
...
Downloads
Documentation
Tests (& Examples)
License
Apache License, Version 2.0 - http://www.apache.org/licenses/LICENSE-2.0.html