What is protobufjs?
The protobufjs npm package provides a comprehensive suite of tools for working with Protocol Buffers (protobuf), a method of serializing structured data. It allows users to encode and decode protobuf messages, generate and work with static code, and handle dynamic message building and parsing.
What are protobufjs's main functionalities?
Loading .proto files
This feature allows users to load .proto files and use the defined protobuf structures within their JavaScript code.
const protobuf = require('protobufjs');
protobuf.load('awesome.proto', function(err, root) {
if (err) throw err;
const AwesomeMessage = root.lookupType('awesomepackage.AwesomeMessage');
// ... use AwesomeMessage
});
Encoding and decoding messages
With protobufjs, users can encode JavaScript objects into binary protobuf format and decode binary messages into JavaScript objects.
const message = AwesomeMessage.create({ awesomeField: 'AwesomeString' });
const buffer = AwesomeMessage.encode(message).finish();
const decodedMessage = AwesomeMessage.decode(buffer);
Reflection and runtime message building
This feature allows users to work with protobuf messages dynamically at runtime using JSON descriptors, without the need for generated static code.
const root = protobuf.Root.fromJSON(jsonDescriptor);
const AwesomeMessage = root.lookupType('awesomepackage.AwesomeMessage');
const errMsg = AwesomeMessage.verify({ awesomeField: 'AwesomeString' });
if (errMsg) throw Error(errMsg);
const message = AwesomeMessage.create({ awesomeField: 'AwesomeString' });
Static code generation
Protobufjs can generate static code from .proto files, which can be used for better performance and type safety.
protobuf.load('awesome.proto', function(err, root) {
if (err) throw err;
protobuf.codegen(root, { keepCase: true }, function(err, output) {
if (err) throw err;
// output will contain the generated static code
});
});
Other packages similar to protobufjs
@apollo/protobufjs
This is a fork of the original protobufjs package with some modifications. It is used within the Apollo tooling ecosystem but generally offers similar functionality to protobufjs.
google-protobuf
This is the official Protocol Buffers runtime library for JavaScript. It is provided by Google and offers similar serialization and deserialization capabilities. However, it may not be as feature-rich or flexible as protobufjs in terms of dynamic message handling and may require more setup for code generation.
pbf
Pbf is a fast, lightweight Protocol Buffers implementation in JavaScript. It focuses on performance and is smaller in size compared to protobufjs. However, it might not offer the same level of functionality, especially in terms of reflection and dynamic message building.
ProtoBuf.js - protobuf for JavaScript
A protobuf implementation on top of ByteBuffer.js including a .proto parser,
reflection, message class building and simple encoding and decoding in plain JavaScript. No compilation step required,
works out of the box on .proto files.
Builder
Probably the core component of ProtoBuf.js. Resolves all type references, performs all the necessary checks and returns
ready to use classes. Can be created from a .proto file or from a JSON definition. The later does not even require the
.proto parser to be included (see: ProtoBuf.noparse.js
).
Example: tests/complex.proto
Install: npm install protobufjs
var ProtoBuf = require("protobufjs");
var builder = ProtoBuf.protoFromFile("tests/complex.proto");
var Game = builder.build("Game");
var Car = Game.Cars.Car;
var car = new Car("Rusty", new Car.Vendor("Iron Inc.", new Car.Vendor.Address("US")), Car.Speed.SUPERFAST);
var car = new Car({
"model": "Rusty",
"vendor": {
"name": "Iron Inc.",
"address": {
"country": "US"
}
},
"speed": "SUPERFAST"
});
var buffer = car.encode();
var socket = ...;
socket.send(buffer.toArrayBuffer());
socket.send(car.toArrayBuffer());
Parser
Compliant with the protobuf parser to the following extend:
-
Required, optional, repeated and packed repeated fields:
message Test {
required int32 a = 1;
optional int32 b = 2 [default=100];
repeated int32 c = 3;
repeated int32 c = 4 [packed=true];
}
-
Data types: int32, uint32, sint32, bool, enum, string, bytes, messages, embedded messages, fixed32, sfixed32, float, double:
message Test {
required int32 a = 1; // Varint encoded
required uint32 b = 2; // Varint encoded
required sint32 c = 3; // Varint zigzag encoded
required bool d = 4; // Varint encoded
enum Priority {
LOW = 1;
MEDIUM = 2;
HIGH = 3;
}
optional Priority e = 5 [default=MEDIUM]; // Varint encoded
required string f = 6; // Varint length delimited
required bytes g = 7; // Varint length delimited
required Embedded h = 8; // Varint length delimited
message Embedded {
repeated int32 a = 1; // Multiple tags
repeated int32 b = 2 [packed=true]; // One tag, length delimited
required fixed32 c = 3; // Fixed 4 bytes
required sfixed32 d = 4; // Fixed 4 bytes zigzag encoded
required float e = 5; // Fixed 4 bytes
required double f = 6; // Fixed 8 bytes
}
}
-
Packages
package My.Game;
message Test {
...
}
message Test2 {
required My.Game.Test test = 1;
}
-
Qualified and fully qualified name resolving:
package My.Game;
message Test {
...
enum Priority {
LOW = 1;
MEDIUM = 2;
HIGH = 3;
}
}
message Test2 {
required .My.Game.Test.Priority priority_fqn = 1 [default=LOW];
required Test.Priority priority_qn = 2 [default=MEDIUM];
}
-
Options on all levels:
option toplevel_1 = 10;
option toplevel_2 = "Hello!";
message Test {
option inmessage = "World!";
optional int32 somenumber = 1 [default=123]; // Actually the only one used
}
Not (yet) supported
- Extensions (what for?), imports (put everything into one builder instead) and services (you roll your own, don't you?).
However, if you need anything of the above, please drop me a note how you'd like to see it implemented. It's just that
I have no idea how to benefit from that and therefore I am not sure how to design it.
Calling the parser on your own
var ProtoBuf = require("protobufjs"),
fs = require("fs"),
util = require("util");
var parser = new ProtoBuf.DotProto.Parser(fs.readFileSync("tests/complex.proto"));
var ast = parser.parse();
console.log(util.inspect(ast, false, null, true));
Encoder
Built into all message classes. Just call YourMessage#encode([buffer])
.
...
var YourMessage = builder.build("YourMessage");
var myMessage = new YourMessage(...);
var byteBuffer = myMessage.encode();
var buffer = byteBuffer.toArrayBuffer();
var buffer = myMessage.toArrayBuffer();
var socket = ...;
socket.send(buffer);
Decoder
Built into all message classes. Just call YourMessage.decode(buffer)
.
...
var YourMessage = builder.build("YourMessage");
var buffer = ...;
var myMessage = YourMessage.decode(buffer);
Downloads
Documentation
Tests (& Examples)
Features
- CommonJS compatible
- RequireJS/AMD compatible
- Shim compatible (include the script, then use
var ProtoBuf = dcodeIO.ProtoBuf;
) - node.js compatible, also available via npm
- Closure Compiler ADVANCED_OPTIMIZATIONS compatible (fully annotated)
- Fully documented using jsdoc3
- Well tested through nodeunit
- ByteBuffer.js is the only production dependency
- Small footprint (even smaller if you use JSON definitions instead of .proto files, because no parser is required)
License
Apache License, Version 2.0 - http://www.apache.org/licenses/LICENSE-2.0.html