github.com/fxamacker/cbor
This is a generic CBOR encoder and decoder. It can encode integers and floats to their smallest forms (like float16) when values fit. Each release passes 375+ tests and 250+ million execs fuzzing with 1100+ CBOR files.
What is CBOR? CBOR (RFC 7049) is a binary data format inspired by JSON and MessagePack. CBOR is used in IETF Internet Standards such as COSE (RFC 8152) and CWT (RFC 8392 CBOR Web Token). Even WebAuthn uses CBOR.
Why this CBOR library? It doesn't crash and it has well-balanced qualities: small, fast, safe and easy. It also supports "preferred serialization" by encoding integers and floats to their smallest forms when values fit.
Small apps. Same programs are 4-9 MB smaller by switching to this library. No code gen and the only imported pkg is x448/float16 which is maintained by the same team as this library.
Small data. The toarray, keyasint, and omitempty struct tags shrink size of Go structs encoded to CBOR. Integers encode to smallest form that fits. Floats can shrink from float64 -> float32 -> float16 if values can round-trip.
Fast. v1.3 became faster than a well-known library that uses unsafe optimizations and code gen. Faster libraries will always exist, but speed is only one factor. This library doesn't use unsafe optimizations or code gen.
Safe and reliable. It prevents crashes on malicious CBOR data by using extensive tests, coverage-guided fuzzing, data validation, and avoiding Go's unsafe pkg. Nested levels for CBOR arrays, maps, and tags are limited to 32.
Easy and saves time. It has the same API as Go's encoding/json when possible. Existing structs don't require changes. Go struct tags like `cbor:"name,omitempty"` and `json:"name,omitempty"` work as expected.
Predefined configs make it easier to comply with standards like Canonical CBOR, CTAP2 Canonical CBOR, etc.
Custom configs can be created by setting individual options. E.g., EncOptions.NaNConvert can be set to NaNConvertNone, NaNConvert7e00, NaNConvertQuiet, or NaNConvertPreserveSignal.
Struct tags like keyasint and toarray make compact CBOR data such as COSE, CWT, and SenML easier to use.
👉 Comparisons • Status • Design Goals • Features • Standards • Fuzzing • Usage • Security Policy • License
Comparisons are between this newer library and a well-known library that had 1,000+ stars before this library was created. Default build settings for each library were used for all comparisons.
This library is safer. Small malicious CBOR messages are rejected quickly before they exhaust system resources.
This library is smaller. Programs like senmlCat can be 4 MB smaller by switching to this library. Programs using more complex CBOR data types can be 9.2 MB smaller.
This library is faster for encoding and decoding CBOR Web Token (CWT). However, speed is only one factor and it can vary depending on data types and sizes. Unlike the other library, this one doesn't use Go's unsafe package or code gen.
The resource intensive codec.CborHandle initialization (in the other library) was placed outside the benchmark loop to make sure their library wasn't penalized.
Doing your own comparisons is highly recommended. Use your most common message sizes and data types.
Version 1.x has:
Each commit passes 375+ tests. Each release also passes 250+ million execs in coverage-guided fuzzing using 1,100+ CBOR files (corpus). See Fuzzing and Code Coverage.
Recent activity:
toarray struct tag to simplify using CBOR arrays.keyasint struct tag to simplify using CBOR maps with int keys.Coming soon: support for CBOR tags (major type 6). After that, options for handling duplicate map keys.
This library is designed to be a generic CBOR encoder and decoder. It was initially created for a WebAuthn (FIDO2) server library, because existing CBOR libraries (in Go) didn't meet certain criteria in 2019.
This library is designed to be:
encoding/json plus keyasint and toarray struct tags.unsafe pkg, coverage >95%, coverage-guided fuzzing, and data validation to avoid crashes on malformed or malicious data.Competing factors are balanced:
unsafe pkg. For speed, use safe optimizations such as caching struct metadata. v1.4 is faster than a well-known library that uses unsafe and code gen.Avoiding unsafe package has benefits. The unsafe package warns:
Packages that import unsafe may be non-portable and are not protected by the Go 1 compatibility guidelines.
All releases prioritize reliability to avoid crashes on decoding malformed CBOR data. See Fuzzing and Coverage.
Features not in Go's standard library are usually not added. However, the toarray struct tag in ugorji/go was too useful to ignore. It was added in v1.3 when a project mentioned they were using it with CBOR to save disk space.
encoding/json plus extra struct tags:
cbor.Encoder writes CBOR to io.Writer.cbor.Decoder reads CBOR from io.Reader.cbor.Marshal writes CBOR to []byte.cbor.Unmarshal reads CBOR from []byte.toarray struct tag allows named struct fields for elements of CBOR arrays.keyasint struct tag allows named struct fields for elements of CBOR maps with int keys.encoding.BinaryMarshaler and encoding.BinaryUnmarshaler interfaces.cbor.RawMessage which can delay CBOR decoding or precompute CBOR encoding.cbor.Marshaler and cbor.Unmarshaler interfaces to allow user-defined types to have custom CBOR encoding and decoding.time.Time as RFC 3339 formatted text string or Unix time.io.LimitReader can be used to limit sizes.encoding/json.)Coming soon: support for CBOR tags (major type 6). After that, options for handling duplicate map keys.
This library implements CBOR as specified in RFC 7049 with minor limitations.
For Go integers, encoder always uses "preferred serialization" which encodes their values to the smallest number of bytes.
Encoder has options that can be set individually to create custom configurations. Easy functions are also provided to create and return modifiable configurations (EncOptions):
EncOptions.Sort:
Encoder has 3 types of options for floating-point data: ShortestFloatMode, InfConvertMode, and NaNConvertMode.
EncOptions.ShortestFloat:
With ShortestFloat16, each floating-point value (including subnormals) can encode float64 -> float32 -> float16 when values can round-trip. Conversions for infinity and NaN use InfConvert and NaNConvert settings.
EncOptions.InfConvert:
EncOptions.NaNConvert:
Float16 conversions use x448/float16 maintained by the same team as this library. All 4+ billion possible conversions are verified to be correct in that library.
Decoder checks for all required well-formedness errors, including all "subkinds" of syntax errors and too little data.
After well-formedness is verified, basic validity errors are handled as follows:
When decoding well-formed CBOR arrays and maps, decoder saves the first error it encounters and continues with the next item. Options to handle this differently may be added in the future.
CBOR tags (type 6) is being added in the next release (milestone v2.0) and is coming soon.
Known limitations:
cbor.UnmarshalTypeError like Go's encoding/json.Undefined (0xf7) value decodes to Go's nil value. Use CBOR Null (0xf6) to round-trip with Go's nil.Like Go's encoding/json, data validation checks the entire message to prevent partially filled (corrupted) data. This library also prevents crashes and resource exhaustion attacks from malicious CBOR data. Use Go's io.LimitReader when decoding very large data to limit size.
Over 375 tests must pass before tagging a release. They include all RFC 7049 examples, bugs found by fuzzing, 2 maliciously crafted CBOR data, and over 87 tests with malformed data.
Code coverage must not fall below 95% when tagging a release. Code coverage is 97.9% (go test -cover) for cbor v1.5 which is among the highest for libraries (in Go) of this type.
Coverage-guided fuzzing must pass 250+ million execs before tagging a release. E.g. v1.4 passed 532+ million execs in coverage-guided fuzzing at the time of release and reached 4+ billion execs 18 days later. Fuzzing uses fxamacker/cbor-fuzz. Default corpus has:
Over 1,100 files (corpus) are used for fuzzing because it includes fuzz-generated corpus.
This project uses Semantic Versioning, so the API is always backwards compatible unless the major version number changes.
The API is the same as encoding/json when possible.
In addition to the API, the keyasint and toarray struct tags are worth knowing. They can reduce programming effort, improve system performance, and reduce the size of serialized data.
package cbor // import "github.com/fxamacker/cbor"
func Marshal(v interface{}, encOpts EncOptions) ([]byte, error)
func Unmarshal(data []byte, v interface{}) error
func Valid(data []byte) (rest []byte, err error)
type Decoder struct{ ... }
func NewDecoder(r io.Reader) *Decoder
func (dec *Decoder) Decode(v interface{}) (err error)
func (dec *Decoder) NumBytesRead() int
type EncOptions struct{ ... }
func CTAP2EncOptions() EncOptions
func CanonicalEncOptions() EncOptions
func CoreDetEncOptions() EncOptions
func PreferredUnsortedEncOptions() EncOptions
type Encoder struct{ ... }
func NewEncoder(w io.Writer, encOpts EncOptions) *Encoder
func (enc *Encoder) Encode(v interface{}) error
func (enc *Encoder) StartIndefiniteByteString() error
func (enc *Encoder) StartIndefiniteTextString() error
func (enc *Encoder) StartIndefiniteArray() error
func (enc *Encoder) StartIndefiniteMap() error
func (enc *Encoder) EndIndefinite() error
type InfConvertMode int
const InfConvertFloat16 InfConvertMode = iota ...
type InvalidUnmarshalError struct{ ... }
type Marshaler interface{ ... }
type NaNConvertMode int
const NaNConvert7e00 NaNConvertMode = iota ...
type RawMessage []byte
type SemanticError struct{ ... }
type ShortestFloatMode int
const ShortestFloatNone ShortestFloatMode = iota ...
type SortMode int
const SortNone SortMode = 0 ...
type SyntaxError struct{ ... }
type UnmarshalTypeError struct{ ... }
type Unmarshaler interface{ ... }
type UnsupportedTypeError struct{ ... }
See API docs for more details.
go get github.com/fxamacker/cbor
Released versions benefit from longer fuzz tests.
👉 Use Go's io.LimitReader when decoding very large data to limit size.
The API is the same as encoding/json when possible:
The keyasint and toarray struct tags make it easy to use compact CBOR message formats. Internet standards often use CBOR arrays and CBOR maps with int keys to save space.
Using named struct fields instead of array elements or maps with int keys makes code more readable and less error prone.
Decoding CWT (CBOR Web Token) using keyasint and toarray struct tags:
// Signed CWT is defined in RFC 8392
type signedCWT struct {
_ struct{} `cbor:",toarray"`
Protected []byte
Unprotected coseHeader
Payload []byte
Signature []byte
}
// Part of COSE header definition
type coseHeader struct {
Alg int `cbor:"1,keyasint,omitempty"`
Kid []byte `cbor:"4,keyasint,omitempty"`
IV []byte `cbor:"5,keyasint,omitempty"`
}
// data is []byte containing signed CWT
var v signedCWT
if err := cbor.Unmarshal(data, &v); err != nil {
return err
}
Encoding CWT (CBOR Web Token) using keyasint and toarray struct tags:
// Use signedCWT struct defined in "Decoding CWT" example.
var v signedCWT
...
if data, err := cbor.Marshal(v, cbor.EncOptions{}); err != nil {
return err
}
Decoding SenML using keyasint struct tag:
// RFC 8428 says, "The data is structured as a single array that
// contains a series of SenML Records that can each contain fields"
type SenMLRecord struct {
BaseName string `cbor:"-2,keyasint,omitempty"`
BaseTime float64 `cbor:"-3,keyasint,omitempty"`
BaseUnit string `cbor:"-4,keyasint,omitempty"`
BaseValue float64 `cbor:"-5,keyasint,omitempty"`
BaseSum float64 `cbor:"-6,keyasint,omitempty"`
BaseVersion int `cbor:"-1,keyasint,omitempty"`
Name string `cbor:"0,keyasint,omitempty"`
Unit string `cbor:"1,keyasint,omitempty"`
Value float64 `cbor:"2,keyasint,omitempty"`
ValueS string `cbor:"3,keyasint,omitempty"`
ValueB bool `cbor:"4,keyasint,omitempty"`
ValueD string `cbor:"8,keyasint,omitempty"`
Sum float64 `cbor:"5,keyasint,omitempty"`
Time float64 `cbor:"6,keyasint,omitempty"`
UpdateTime float64 `cbor:"7,keyasint,omitempty"`
}
// data is a []byte containing SenML
var v []SenMLRecord
if err := cbor.Unmarshal(data, &v); err != nil {
return err
}
Encoding SenML using keyasint struct tag and ShortestFloat16 encoding option:
// use SenMLRecord struct defined in "Decoding SenML" example
var v []SenMLRecord
...
if data, err := cbor.Marshal(v, cbor.EncOptions{ShortestFloat: cbor.ShortestFloat16}); err != nil {
return err
}
Decoding:
// create a decoder
dec := cbor.NewDecoder(reader)
// decode into empty interface
var i interface{}
err = dec.Decode(&i)
// decode into struct
var stru ExampleStruct
err = dec.Decode(&stru)
// decode into map
var m map[string]string
err = dec.Decode(&m)
// decode into primitive
var f float32
err = dec.Decode(&f)
Encoding:
// create an encoder with canonical CBOR encoding enabled
enc := cbor.NewEncoder(writer, cbor.CanonicalEncOptions())
// encode struct
err = enc.Encode(stru)
// encode map
err = enc.Encode(m)
// encode primitive
err = enc.Encode(f)
Encoding indefinite length array:
enc := cbor.NewEncoder(writer, cbor.EncOptions{})
// start indefinite length array encoding
err = enc.StartIndefiniteArray()
// encode array element
err = enc.Encode(1)
// encode array element
err = enc.Encode([]int{2, 3})
// start nested indefinite length array as array element
err = enc.StartIndefiniteArray()
// encode nested array element
err = enc.Encode(4)
// encode nested array element
err = enc.Encode(5)
// end nested indefinite length array
err = enc.EndIndefinite()
// end indefinite length array
err = enc.EndIndefinite()
More examples.
Go structs are faster than maps with string keys:
Go structs with keyasint struct tag are faster than maps with integer keys:
Go structs with toarray struct tag are faster than slice:
Doing your own benchmarks is highly recommended. Use your most common message sizes and data types.
See Benchmarks for fxamacker/cbor.
This project has adopted the Contributor Covenant Code of Conduct. Contact faye.github@gmail.com with any questions or comments.
Please refer to How to Contribute.
For v1, security fixes are provided only for the latest released version since the API won't break compatibility.
To report security vulnerabilities, please email faye.github@gmail.com and allow time for the problem to be resolved before reporting it to the public.
Phrases like "no crashes" or "doesn't crash" mean there are no known crash bugs in the latest version based on results of unit tests and coverage-guided fuzzing. It doesn't imply the software is 100% bug-free or 100% invulnerable to all known and unknown attacks.
Please read the license for additional disclaimers and terms.
Copyright (c) Faye Amacker. All rights reserved.
Licensed under the MIT License.
👉 Comparisons • Status • Design Goals • Features • Standards • Fuzzing • Usage • Security Policy • License
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.