Geobuf
Geobuf is a compact binary encoding for geographic data.
Geobuf provides nearly lossless compression of GeoJSON data
into protocol buffers.
Advantages over using GeoJSON alone:
- Very compact: typically makes GeoJSON 6-8 times smaller.
- 2-2.5x smaller even when comparing gzipped sizes.
- Very fast encoding and decoding — even faster than native JSON parse/stringify.
- Can accommodate any GeoJSON data, including extensions with arbitrary properties.
The encoding format also potentially allows:
- Easy incremental parsing — get features out as you read them,
without the need to build in-memory representation of the whole data.
- Partial reads — read only the parts you actually need, skipping the rest.
Think of this as an attempt to design a simple, modern Shapefile successor
that works seamlessly with GeoJSON.
Unlike Mapbox Vector Tiles,
it aims for nearly lossless compression of datasets — without tiling, projecting coordinates,
flattening geometries or stripping properties.
Note that the encoding schema is not stable yet —
it may still change as we get community feedback and discover new ways to improve it.
"Nearly" lossless means coordinates are encoded with precision of 6 digits after the decimal point (about 10cm).
Sample compression sizes
Data | JSON | JSON (gz) | Geobuf | Geobuf (gz) |
---|
US zip codes | 101.85 MB | 26.67 MB | 12.24 MB | 10.48 MB |
Idaho counties | 10.92 MB | 2.57 MB | 1.37 MB | 1.17 MB |
API
encode
var buffer = geobuf.encode(geojson, new Pbf());
Given a GeoJSON object and a Pbf object to write to,
returns a Geobuf as UInt8Array
array of bytes.
In Node@4.5.0 or later, you can use Buffer.from
to convert back to a buffer.
decode
var geojson = geobuf.decode(new Pbf(data));
Given a Pbf object with Geobuf data, return a GeoJSON object. When loading Geobuf data over XMLHttpRequest
, you need to set responseType
to arraybuffer
.
Install
Node and Browserify:
npm install geobuf
Browser build CDN links:
Building locally:
npm install
npm run build-dev
npm run build-min
Command Line
npm install -g geobuf
Installs these nifty binaries:
geobuf2json
: turn Geobuf from stdin
or specified file to GeoJSON on stdout
json2geobuf
: turn GeoJSON from stdin
or specified file to Geobuf on stdout
shp2geobuf
: given a Shapefile filename, send Geobuf on stdout
json2geobuf data.json > data.pbf
shp2geobuf myshapefile > data.pbf
geobuf2json data.pbf > data.json
Note that for big files, geobuf2json
command can be pretty slow, but the bottleneck is not the decoding,
but the native JSON.stringify
on the decoded object to pipe it as a string to stdout
.
On some files, this step may take 40 times more time than actual decoding.
See Also
- geojsonp — the prototype that led to this project
- pygeobuf — Python implementation of Geobuf
- twkb — a geospatial binary encoding that doesn't support topology
and doesn't encode any non-geographic properties besides
id
- vector-tile-spec
- topojson — an extension of GeoJSON that supports topology
- WKT and WKB — popular in databases
- EWKB — a popular superset of WKB