github.com/Salvionied/cbor-cardano
Package cbor is a modern CBOR codec (RFC 8949 & RFC 7049) with CBOR tags, Go struct tags (toarray/keyasint/omitempty), Core Deterministic Encoding, CTAP2, Canonical CBOR, float64->32->16, and duplicate map key detection. Encoding options allow "preferred serialization" by encoding integers and floats to their smallest forms (e.g. float16) when values fit. Struct tags like "keyasint", "toarray" and "omitempty" make CBOR data smaller and easier to use with structs. For example, "toarray" tag makes struct fields encode to CBOR array elements. And "keyasint" makes a field encode to an element of CBOR map with specified int key. Latest docs can be viewed at https://github.com/fxamacker/cbor#cbor-library-in-go The Quick Start guide is at https://github.com/fxamacker/cbor#quick-start Function signatures identical to encoding/json include: Standard interfaces include: Custom encoding and decoding is possible by implementing standard interfaces for user-defined Go types. Codec functions are available at package-level (using defaults options) or by creating modes from options at runtime. "Mode" in this API means definite way of encoding (EncMode) or decoding (DecMode). EncMode and DecMode interfaces are created from EncOptions or DecOptions structs. Modes use immutable options to avoid side-effects and simplify concurrency. Behavior of modes won't accidentally change at runtime after they're created. Modes are intended to be reused and are safe for concurrent use. EncMode and DecMode Interfaces Using Default Encoding Mode Using Default Decoding Mode Creating and Using Encoding Modes Predefined Encoding Options: https://github.com/fxamacker/cbor#predefined-encoding-options Encoding Options: https://github.com/fxamacker/cbor#encoding-options Decoding Options: https://github.com/fxamacker/cbor#decoding-options Struct tags like `cbor:"name,omitempty"` and `json:"name,omitempty"` work as expected. If both struct tags are specified then `cbor` is used. Struct tags like "keyasint", "toarray", and "omitempty" make it easy to use very compact formats like COSE and CWT (CBOR Web Tokens) with structs. For example, "toarray" makes struct fields encode to array elements. And "keyasint" makes struct fields encode to elements of CBOR map with int keys. https://raw.githubusercontent.com/fxamacker/images/master/cbor/v2.0.0/cbor_easy_api.png Struct tags are listed at https://github.com/fxamacker/cbor#struct-tags-1 Over 375 tests are included in this package. Cover-guided fuzzing is handled by fxamacker/cbor-fuzz.
Readme
fxamacker/cbor is a modern CBOR codec in Go. It's like encoding/json for CBOR with time-saving features. It balances security, usability, speed, data size, program size, and other competing factors.
Features include CBOR tags, duplicate map key detection, float64→32→16, and Go struct tags (toarray, keyasint, omitempty). API is close to encoding/json plus predefined CBOR options like Core Deterministic Encoding, Preferred Serialization, CTAP2, etc.
Using CBOR Preferred Serialization with Go struct tags (toarray, keyasint, omitempty) reduces programming effort and creates smaller encoded data size.
There are 1276 repositories that depend on fxamacker/cbor/v2. Additional 155 repositories are using version 1.x of this CBOR codec (please upgrade to v2).
fxamacker/cbor is used by Arm Ltd., Berlin Institute of Health at Charité, Chainlink, ConsenSys, Dapper Labs, Duo Labs (cisco), EdgeX Foundry, Mozilla, National Cybersecurity Agency of France (govt), Netherlands (govt), Oasis Labs, Tailscale, Taurus SA, Teleport, TIBCO, and others.
Microsoft Corporation had NCC Group conduct a security assessment (PDF) which includes portions of this library in its scope.
fxamacker/cbor has 98% coverage and is fuzz tested.
Install with go get github.com/fxamacker/cbor/v2 and import "github.com/fxamacker/cbor/v2".
See Quick Start to save time.
CBOR is a concise binary data format inspired by JSON and MessagePack. CBOR is defined in RFC 8949 (December 2020) which obsoletes RFC 7049 (October 2013).
CBOR is an Internet Standard by IETF. It's used in other standards like WebAuthn by W3C, COSE (RFC 8152), CWT (RFC 8392), CDDL (RFC 8610) and more.
Reasons for choosing CBOR vary by project. Some projects replaced protobuf, encoding/json, encoding/gob, etc. with CBOR. For example, by replacing protobuf with CBOR in gRPC.
fxamacker/cbor balances competing factors such as speed, size, safety, usability, maintainability, and etc.
Killer features include Go struct tags like toarray, keyasint, etc. They reduce encoded data size, improve speed, and reduce programming effort. For example, toarray automatically translates a Go struct to/from a CBOR array.
Modern CBOR features include Core Deterministic Encoding and Preferred Encoding. Other features include CBOR tags, big.Int, float64→32→16, an API like encoding/json, and more.
Security features include the option to detect duplicate map keys and options to set various max limits. And it's designed to make concurrent use of CBOR options easy and free from side-effects.
To prevent crashes, it has been fuzz-tested since before release 1.0 and code coverage is kept above 98%.
For portability and safety, it avoids using unsafe, which makes it portable and protected by Go1's compatibility guidelines.
For performance, it uses safe optimizations. When used properly, fxamacker/cbor can be faster than CBOR codecs that rely on unsafe. However, speed is only one factor and should be considered together with other competing factors.
fxamacker/cbor is secure. It rejects malformed CBOR data and has an option to detect duplicate map keys. It doesn't crash when decoding bad CBOR data. It has extensive tests, coverage-guided fuzzing, data validation, and avoids Go's unsafe package.
Decoding 9 or 10 bytes of malformed CBOR data shouldn't exhaust memory. For example,
[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}
| Decode bad 10 bytes to interface{} | Decode bad 10 bytes to []byte | |
|---|---|---|
| fxamacker/cbor 1.0-2.3 | 49.44 ns/op, 24 B/op, 2 allocs/op* | 51.93 ns/op, 32 B/op, 2 allocs/op* |
| ugorji/go 1.2.6 | ⚠️ 45021 ns/op, 262852 B/op, 7 allocs/op | 💥 runtime: out of memory: cannot allocate |
| ugorji/go 1.1-1.1.7 | 💥 runtime: out of memory: cannot allocate | 💥 runtime: out of memory: cannot allocate |
*Speed and memory are for latest codec version listed in the row (compiled with Go 1.17.5).
fxamacker/cbor CBOR safety settings include: MaxNestedLevels, MaxArrayElements, MaxMapPairs, and IndefLength.
For more info, see:
fxamacker/cbor is fast without sacrificing security. It can be faster than libraries relying on unsafe package.
Click to expand:
fxamacker/cbor produces smaller programs without sacrificing features.
fxamacker/cbor 2.3.0 (not using unsafe) is faster than ugorji/go 1.2.6 (using unsafe).
benchstat results/bench-ugorji-go-count20.txt results/bench-fxamacker-cbor-count20.txt
name old time/op new time/op delta
DecodeCWTClaims-8 1.08µs ± 0% 0.67µs ± 0% -38.10% (p=0.000 n=16+20)
DecodeCOSE/128-Bit_Symmetric_Key-8 715ns ± 0% 501ns ± 0% -29.97% (p=0.000 n=20+19)
DecodeCOSE/256-Bit_Symmetric_Key-8 722ns ± 0% 507ns ± 0% -29.72% (p=0.000 n=19+18)
DecodeCOSE/ECDSA_P256_256-Bit_Key-8 1.11µs ± 0% 0.83µs ± 0% -25.27% (p=0.000 n=19+20)
DecodeWebAuthn-8 880ns ± 0% 727ns ± 0% -17.31% (p=0.000 n=18+20)
EncodeCWTClaims-8 785ns ± 0% 388ns ± 0% -50.51% (p=0.000 n=20+20)
EncodeCOSE/128-Bit_Symmetric_Key-8 973ns ± 0% 433ns ± 0% -55.45% (p=0.000 n=20+19)
EncodeCOSE/256-Bit_Symmetric_Key-8 974ns ± 0% 435ns ± 0% -55.37% (p=0.000 n=20+19)
EncodeCOSE/ECDSA_P256_256-Bit_Key-8 1.14µs ± 0% 0.55µs ± 0% -52.10% (p=0.000 n=19+19)
EncodeWebAuthn-8 564ns ± 0% 450ns ± 1% -20.18% (p=0.000 n=18+20)
name old alloc/op new alloc/op delta
DecodeCWTClaims-8 744B ± 0% 160B ± 0% -78.49% (p=0.000 n=20+20)
DecodeCOSE/128-Bit_Symmetric_Key-8 792B ± 0% 232B ± 0% -70.71% (p=0.000 n=20+20)
DecodeCOSE/256-Bit_Symmetric_Key-8 816B ± 0% 256B ± 0% -68.63% (p=0.000 n=20+20)
DecodeCOSE/ECDSA_P256_256-Bit_Key-8 905B ± 0% 344B ± 0% -61.99% (p=0.000 n=20+20)
DecodeWebAuthn-8 1.56kB ± 0% 0.99kB ± 0% -36.41% (p=0.000 n=20+20)
EncodeCWTClaims-8 1.35kB ± 0% 0.18kB ± 0% -86.98% (p=0.000 n=20+20)
EncodeCOSE/128-Bit_Symmetric_Key-8 1.95kB ± 0% 0.22kB ± 0% -88.52% (p=0.000 n=20+20)
EncodeCOSE/256-Bit_Symmetric_Key-8 1.95kB ± 0% 0.24kB ± 0% -87.70% (p=0.000 n=20+20)
EncodeCOSE/ECDSA_P256_256-Bit_Key-8 1.95kB ± 0% 0.32kB ± 0% -83.61% (p=0.000 n=20+20)
EncodeWebAuthn-8 1.30kB ± 0% 1.09kB ± 0% -16.56% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
DecodeCWTClaims-8 6.00 ± 0% 6.00 ± 0% ~ (all equal)
DecodeCOSE/128-Bit_Symmetric_Key-8 4.00 ± 0% 4.00 ± 0% ~ (all equal)
DecodeCOSE/256-Bit_Symmetric_Key-8 4.00 ± 0% 4.00 ± 0% ~ (all equal)
DecodeCOSE/ECDSA_P256_256-Bit_Key-8 7.00 ± 0% 7.00 ± 0% ~ (all equal)
DecodeWebAuthn-8 5.00 ± 0% 5.00 ± 0% ~ (all equal)
EncodeCWTClaims-8 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=20+20)
EncodeCOSE/128-Bit_Symmetric_Key-8 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=20+20)
EncodeCOSE/256-Bit_Symmetric_Key-8 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=20+20)
EncodeCOSE/ECDSA_P256_256-Bit_Key-8 6.00 ± 0% 2.00 ± 0% -66.67% (p=0.000 n=20+20)
EncodeWebAuthn-8 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=20+20)
Benchmarks used Go 1.17.5, linux_amd64, and data from RFC 8392 Appendix A.1. Default build options were used for all CBOR libraries. Library init code was put outside the benchmark loop for all libraries compared.
fxamacker/cbor is easy to use. It provides standard API and interfaces.
Standard API. Function signatures identical to encoding/json include:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, and (*Decoder).Decode.
Standard Interfaces. Custom encoding and decoding is handled by implementing:
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler.
Predefined Encoding Options. Encoding options are easy to use and are customizable.
func CoreDetEncOptions() EncOptions {} // RFC 8949 Core Deterministic Encoding
func PreferredUnsortedEncOptions() EncOptions {} // RFC 8949 Preferred Serialization
func CanonicalEncOptions() EncOptions {} // RFC 7049 Canonical CBOR
func CTAP2EncOptions() EncOptions {} // FIDO2 CTAP2 Canonical CBOR
fxamacker/cbor designed to simplify concurrency. CBOR options can be used without creating unintended runtime side-effects.
fxamacker/cbor provides Go struct tags like toarray and keyasint to save time and reduce encoded size of data.
fxamacker/cbor is a full-featured CBOR encoder and decoder.
| CBOR Feature | Description | |
|---|---|---|
| ☑️ | CBOR tags | API supports built-in and user-defined tags. |
| ☑️ | Preferred serialization | Integers encode to fewest bytes. Optional float64 → float32 → float16. |
| ☑️ | Map key sorting | Unsorted, length-first (Canonical CBOR), and bytewise-lexicographic (CTAP2). |
| ☑️ | Duplicate map keys | Always forbid for encoding and option to allow/forbid for decoding. |
| ☑️ | Indefinite length data | Option to allow/forbid for encoding and decoding. |
| ☑️ | Well-formedness | Always checked and enforced. |
| ☑️ | Basic validity checks | Check UTF-8 validity and optionally check duplicate map keys. |
| ☑️ | Security considerations | Prevent integer overflow and resource exhaustion (RFC 8949 Section 10). |
fxamacker/cbor supports Go 1.12 and newer versions. Init the Go module, go get v2, and begin coding.
go mod init github.com/my_name/my_repo
go get github.com/fxamacker/cbor/v2
import "github.com/fxamacker/cbor/v2" // imports as cbor
🛡️ Use Go's io.LimitReader to limit size when decoding very large or indefinite size data.
Import using "/v2" like this: import "github.com/fxamacker/cbor/v2", and
it will import version 2.x as package "cbor" (when using Go modules).
Functions with identical signatures to encoding/json include:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode.
Default Mode
If default options are acceptable, package level functions can be used for encoding and decoding.
b, err := cbor.Marshal(v) // encode v to []byte b
err := cbor.Unmarshal(b, &v) // decode []byte b to v
encoder := cbor.NewEncoder(w) // create encoder with io.Writer w
decoder := cbor.NewDecoder(r) // create decoder with io.Reader r
Modes
If you need to use options or CBOR tags, then you'll want to create a mode.
"Mode" means defined way of encoding or decoding -- it links the standard API to your CBOR options and CBOR tags. This way, you don't pass around options and the API remains identical to encoding/json.
EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.
For example, em, err := cbor.EncOptions{...}.EncMode() or em, err := cbor.CanonicalEncOptions().EncMode().
EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are reusable, safe for concurrent use, and allow fast parallelism.
Creating and Using Encoding Modes
💡 Avoid using init(). For best performance, reuse EncMode and DecMode after creating them.
Most apps will probably create one EncMode and DecMode before init(). There's no limit and each can use different options.
// Create EncOptions using either struct literal or a function.
opts := cbor.CanonicalEncOptions()
// If needed, modify opts. For example: opts.Time = cbor.TimeUnix
// Create reusable EncMode interface with immutable options, safe for concurrent use.
em, err := opts.EncMode()
// Use EncMode like encoding/json, with same function signatures.
b, err := em.Marshal(v) // encode v to []byte b
encoder := em.NewEncoder(w) // create encoder with io.Writer w
err := encoder.Encode(v) // encode v to io.Writer w
Both em.Marshal(v) and encoder.Encode(v) use encoding options specified during creation of encoding mode em.
Creating Modes With CBOR Tags
A TagSet is used to specify CBOR tags.
em, err := opts.EncMode() // no tags
em, err := opts.EncModeWithTags(ts) // immutable tags
em, err := opts.EncModeWithSharedTags(ts) // mutable shared tags
TagSet and all modes using it are safe for concurrent use. Equivalent API is available for DecMode.
Predefined Encoding Options
func CoreDetEncOptions() EncOptions {} // RFC 8949 Core Deterministic Encoding
func PreferredUnsortedEncOptions() EncOptions {} // RFC 8949 Preferred Serialization
func CanonicalEncOptions() EncOptions {} // RFC 7049 Canonical CBOR
func CTAP2EncOptions() EncOptions {} // FIDO2 CTAP2 Canonical CBOR
The empty curly braces prevent a syntax highlighting bug on GitHub, please ignore them.
Struct Tags (keyasint, toarray, omitempty)
The keyasint, toarray, and omitempty struct tags make it easy to use compact CBOR message formats. Internet standards often use CBOR arrays and CBOR maps with int keys to save space.
The following sections provide more info:
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
Many function signatures are identical to encoding/json, including:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode.
RawMessage can be used to delay CBOR decoding or precompute CBOR encoding, like encoding/json.
Standard interfaces allow user-defined types to have custom CBOR encoding and decoding. They include:
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler.
Marshaler and Unmarshaler interfaces are satisfied by MarshalCBOR and UnmarshalCBOR functions using same params and return types as Go's MarshalJSON and UnmarshalJSON.
Support "cbor" and "json" keys in Go's struct tags. If both are specified for the same field, then "cbor" is used.
omitempty omits (ignores) field if value is empty, like encoding/json.- always omits (ignores) field, like encoding/json.keyasint treats fields as elements of CBOR maps with specified int key.toarray treats fields as elements of CBOR arrays.See Struct Tags for more info.
There are three categories of CBOR tags:
Default built-in CBOR tags currently include tag numbers 0 (Standard Date/Time), 1 (Epoch Date/Time), 2 (Unsigned Bignum), 3 (Negative Bignum), 55799 (Self-Described CBOR).
Optional built-in CBOR tags may be provided in the future via build flags or optional package(s) to help reduce bloat.
User-defined CBOR tags are easy by using TagSet to associate tag numbers to user-defined Go types.
Preferred serialization encodes integers and floating-point values using the fewest bytes possible.
The combination of preferred serialization and struct tags (toarray, keyasint, omitempty) allows very compact data size.
Easy-to-use functions (no params) return preset EncOptions struct:
CanonicalEncOptions, CTAP2EncOptions, CoreDetEncOptions, PreferredUnsortedEncOptions
Integers always encode to the shortest form that preserves value. By default, time values are encoded without tags.
Encoding of other data types and map key sort order are determined by encoder options.
| EncOptions | Available Settings (defaults listed first) |
|---|---|
| Sort | SortNone, SortLengthFirst, SortBytewiseLexical Aliases: SortCanonical, SortCTAP2, SortCoreDeterministic |
| Time | TimeUnix, TimeUnixMicro, TimeUnixDynamic, TimeRFC3339, TimeRFC3339Nano |
| TimeTag | EncTagNone, EncTagRequired |
| ShortestFloat | ShortestFloatNone, ShortestFloat16 |
| BigIntConvert | BigIntConvertShortest, BigIntConvertNone |
| InfConvert | InfConvertFloat16, InfConvertNone |
| NaNConvert | NaNConvert7e00, NaNConvertNone, NaNConvertQuiet, NaNConvertPreserveSignal |
| IndefLength | IndefLengthAllowed, IndefLengthForbidden |
| TagsMd | TagsAllowed, TagsForbidden |
See Options section for details about each setting.
| DecOptions | Available Settings (defaults listed first) |
|---|---|
| TimeTag | DecTagIgnored, DecTagOptional, DecTagRequired |
| DupMapKey | DupMapKeyQuiet, DupMapKeyEnforcedAPF |
| IntDec | IntDecConvertNone, IntDecConvertSigned |
| IndefLength | IndefLengthAllowed, IndefLengthForbidden |
| TagsMd | TagsAllowed, TagsForbidden |
| ExtraReturnErrors | ExtraDecErrorNone, ExtraDecErrorUnknownField |
| MaxNestedLevels | 32, can be set to [4, 65535] |
| MaxArrayElements | 131072, can be set to [16, 2147483647] |
| MaxMapPairs | 131072, can be set to [16, 2147483647] |
See Options section for details about each setting.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
This library is a full-featured generic CBOR (RFC 8949) encoder and decoder. Notable CBOR features include:
| CBOR Feature | Description | |
|---|---|---|
| ☑️ | CBOR tags | API supports built-in and user-defined tags. |
| ☑️ | Preferred serialization | Integers encode to fewest bytes. Optional float64 → float32 → float16. |
| ☑️ | Map key sorting | Unsorted, length-first (Canonical CBOR), and bytewise-lexicographic (CTAP2). |
| ☑️ | Duplicate map keys | Always forbid for encoding and option to allow/forbid for decoding. |
| ☑️ | Indefinite length data | Option to allow/forbid for encoding and decoding. |
| ☑️ | Well-formedness | Always checked and enforced. |
| ☑️ | Basic validity checks | Check UTF-8 validity and optionally check duplicate map keys. |
| ☑️ | Security considerations | Prevent integer overflow and resource exhaustion (RFC 8949 Section 10). |
See the Features section for list of Encoding Options and Decoding Options.
Known limitations are noted in the Limitations section.
Go nil values for slices, maps, pointers, etc. are encoded as CBOR null. Empty slices, maps, etc. are encoded as empty CBOR arrays and maps.
Decoder checks for all required well-formedness errors, including all "subkinds" of syntax errors and too little data.
After well-formedness is verified, basic validity errors are handled as follows:
When decoding well-formed CBOR arrays and maps, decoder saves the first error it encounters and continues with the next item. Options to handle this differently may be added in the future.
By default, decoder treats time values of floating-point NaN and Infinity as if they are CBOR Null or CBOR Undefined.
See Options section for detailed settings or Features section for a summary of options.
Click to expand topic:
This library provides options for fast detection and rejection of duplicate map keys based on applying a Go-specific data model to CBOR's extended generic data model in order to determine duplicate vs distinct map keys. Detection relies on whether the CBOR map key would be a duplicate "key" when decoded and applied to the user-provided Go map or struct.
DupMapKeyQuiet turns off detection of duplicate map keys. It tries to use a "keep fastest" method by choosing either "keep first" or "keep last" depending on the Go data type.
DupMapKeyEnforcedAPF enforces detection and rejection of duplidate map keys. Decoding stops immediately and returns DupMapKeyError when the first duplicate key is detected. The error includes the duplicate map key and the index number.
APF suffix means "Allow Partial Fill" so the destination map or struct can contain some decoded values at the time of error. It is the caller's responsibility to respond to the DupMapKeyError by discarding the partially filled result if that's required by their protocol.
This library checks tag validity for built-in tags (currently tag numbers 0, 1, 2, 3, and 55799):
Unknown tag data items (not tag number 0, 1, 2, 3, or 55799) are handled in two ways:
cbor.Tag data type, which contains tag number and tag content. The tag content will be decoded into the default Go data type for the CBOR data type.Decoder also has an option to forbid tag data items (treat any tag data item as error) which is specified by protocols such as CTAP2 Canonical CBOR.
For more information, see decoding options and tag options.
If any of these limitations prevent you from using this library, please open an issue along with a link to your project.
Undefined (0xf7) value decodes to Go's nil value. CBOR Null (0xf6) more closely matches Go's nil.simple values that are unassigned/reserved by IANA are not fully supported until PR #370.io.LimitReader should be used to limit size.⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
Many function signatures are identical to Go's encoding/json, such as:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, and (*Decoder).Decode.
Interfaces identical or comparable to Go's encoding, encoding/json, or encoding/gob include:
Marshaler, Unmarshaler, BinaryMarshaler, and BinaryUnmarshaler.
Like encoding/json, RawMessage can be used to delay CBOR decoding or precompute CBOR encoding.
"Mode" in this API means defined way of encoding or decoding -- it links the standard API to CBOR options and CBOR tags.
EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.
For example, em, err := cbor.EncOptions{...}.EncMode() or em, err := cbor.CanonicalEncOptions().EncMode().
EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are intended to be reused and are safe for concurrent use.
API for Default Mode
If default options are acceptable, then you don't need to create EncMode or DecMode.
Marshal(v interface{}) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
Unmarshal(data []byte, v interface{}) error
NewDecoder(r io.Reader) *Decoder
API for Creating & Using Encoding Modes
// EncMode interface uses immutable options and is safe for concurrent use.
type EncMode interface {
Marshal(v interface{}) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
EncOptions() EncOptions // returns copy of options
}
// EncOptions specifies encoding options.
type EncOptions struct {
...
}
// EncMode returns an EncMode interface created from EncOptions.
func (opts EncOptions) EncMode() (EncMode, error) {}
// EncModeWithTags returns EncMode with options and tags that are both immutable.
func (opts EncOptions) EncModeWithTags(tags TagSet) (EncMode, error) {}
// EncModeWithSharedTags returns EncMode with immutable options and mutable shared tags.
func (opts EncOptions) EncModeWithSharedTags(tags TagSet) (EncMode, error) {}
The empty curly braces prevent a syntax highlighting bug, please ignore them.
API for Predefined Encoding Options
func CoreDetEncOptions() EncOptions {} // RFC 8949 Core Deterministic Encoding
func PreferredUnsortedEncOptions() EncOptions {} // RFC 8949 Preferred Serialization
func CanonicalEncOptions() EncOptions {} // RFC 7049 Canonical CBOR
func CTAP2EncOptions() EncOptions {} // FIDO2 CTAP2 Canonical CBOR
API for Creating & Using Decoding Modes
// DecMode interface uses immutable options and is safe for concurrent use.
type DecMode interface {
Unmarshal(data []byte, v interface{}) error
NewDecoder(r io.Reader) *Decoder
DecOptions() DecOptions // returns copy of options
}
// DecOptions specifies decoding options.
type DecOptions struct {
...
}
// DecMode returns a DecMode interface created from DecOptions.
func (opts DecOptions) DecMode() (DecMode, error) {}
// DecModeWithTags returns DecMode with options and tags that are both immutable.
func (opts DecOptions) DecModeWithTags(tags TagSet) (DecMode, error) {}
// DecModeWithSharedTags returns DecMode with immutable options and mutable shared tags.
func (opts DecOptions) DecModeWithSharedTags(tags TagSet) (DecMode, error) {}
The empty curly braces prevent a syntax highlighting bug, please ignore them.
API for Using CBOR Tags
TagSet can be used to associate user-defined Go type(s) to tag number(s). It's also used to create EncMode or DecMode. For example, em := EncOptions{...}.EncModeWithTags(ts) or em := EncOptions{...}.EncModeWithSharedTags(ts). This allows every standard API exported by em (like Marshal and NewEncoder) to use the specified tags automatically.
Tag and RawTag can be used to encode/decode a tag number with a Go value, but TagSet is generally recommended.
type TagSet interface {
// Add adds given tag number(s), content type, and tag options to TagSet.
Add(opts TagOptions, contentType reflect.Type, num uint64, nestedNum ...uint64) error
// Remove removes given tag content type from TagSet.
Remove(contentType reflect.Type)
}
Tag and RawTag types can also be used to encode/decode tag number with Go value.
type Tag struct {
Number uint64
Content interface{}
}
type RawTag struct {
Number uint64
Content RawMessage
}
See API docs (godoc.org) for more details and more functions. See Usage section for usage and code examples.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
Struct tags, decoding options, and encoding options.
This library supports both "cbor" and "json" key for some (not all) struct tags. If "cbor" and "json" keys are both present for the same field, then "cbor" key will be used.
| Key | Format Str | Scope | Description |
|---|---|---|---|
| cbor or json | "myName" | field | Name of field to use such as "myName", etc. like encoding/json. |
| cbor or json | ",omitempty" | field | Omit (ignore) this field if value is empty, like encoding/json. |
| cbor or json | "-" | field | Omit (ignore) this field always, like encoding/json. |
| cbor | ",keyasint" | field | Treat field as an element of CBOR map with specified int as key. |
| cbor | ",toarray" | struct | Treat each field as an element of CBOR array. This automatically disables "omitempty" and "keyasint" for all fields in the struct. |
The "keyasint" struct tag requires an integer key to be specified:
type myStruct struct {
MyField int64 `cbor:"-1,keyasint,omitempty'`
OurField string `cbor:"0,keyasint,omitempty"`
FooField Foo `cbor:"5,keyasint,omitempty"`
BarField Bar `cbor:"hello,omitempty"`
...
}
The "toarray" struct tag requires a special field "_" (underscore) to indicate "toarray" applies to the entire struct:
type myStruct struct {
_ struct{} `cbor:",toarray"`
MyField int64
OurField string
...
}
Click to expand:
| DecOptions.TimeTag | Description |
|---|---|
| DecTagIgnored (default) | Tag numbers are ignored (if present) for time values. |
| DecTagOptional | Tag numbers are only checked for validity if present for time values. |
| DecTagRequired | Tag numbers must be provided for time values except for CBOR Null and CBOR Undefined. |
The following CBOR time values are decoded as Go's "zero time instant":
Go's time package provides IsZero function, which reports whether t represents "zero time instant"
(January 1, year 1, 00:00:00 UTC).
| DecOptions.DupMapKey | Description |
|---|---|
| DupMapKeyQuiet (default) | turns off detection of duplicate map keys. It uses a "keep fastest" method by choosing either "keep first" or "keep last" depending on the Go data type. |
| DupMapKeyEnforcedAPF | enforces detection and rejection of duplidate map keys. Decoding stops immediately and returns DupMapKeyError when the first duplicate key is detected. The error includes the duplicate map key and the index number. |
DupMapKeyEnforcedAPF uses "Allow Partial Fill" so the destination map or struct can contain some decoded values at the time of error. Users can respond to the DupMapKeyError by discarding the partially filled result if that's required by their protocol.
| DecOptions.IntDec | Description |
|---|---|
| IntDecConvertNone (default) | When decoding to Go interface{}, CBOR positive int (major type 0) decode to uint64 value, and CBOR negative int (major type 1) decode to int64 value. |
| IntDecConvertSigned | When decoding to Go interface{}, CBOR positive/negative int (major type 0 and 1) decode to int64 value. |
If IntDecConvertedSigned is used and value overflows int64, UnmarshalTypeError is returned.
| DecOptions.IndefLength | Description |
|---|---|
| IndefLengthAllowed (default) | allow indefinite length data |
| IndefLengthForbidden | forbid indefinite length data |
| DecOptions.TagsMd | Description |
|---|---|
| TagsAllowed (default) | allow CBOR tags (major type 6) |
| TagsForbidden | forbid CBOR tags (major type 6) |
| DecOptions.ExtraReturnErrors | Description |
|---|---|
| ExtraDecErrorNone (default) | no extra decoding errors. E.g. ignore unknown fields if encountered. |
| ExtraDecErrorUnknownField | return error if unknown field is encountered |
| DecOptions.MaxNestedLevels | Description |
|---|---|
| 32 (default) | allowed setting is [4, 65535] |
| DecOptions.MaxArrayElements | Description |
|---|---|
| 131072 (default) | allowed setting is [16, 2147483647] |
| DecOptions.MaxMapPairs | Description |
|---|---|
| 131072 (default) | allowed setting is [16, 2147483647] |
Integers always encode to the shortest form that preserves value. Encoding of other data types and map key sort order are determined by encoding options.
These functions are provided to create and return a modifiable EncOptions struct with predefined settings.
| Predefined EncOptions | Description |
|---|---|
| CanonicalEncOptions() | Canonical CBOR (RFC 7049 Section 3.9). |
| CTAP2EncOptions() | CTAP2 Canonical CBOR (FIDO2 CTAP2). |
| PreferredUnsortedEncOptions() | Unsorted, encode float64->float32->float16 when values fit, NaN values encoded as float16 0x7e00. |
| CoreDetEncOptions() | PreferredUnsortedEncOptions() + map keys are sorted bytewise lexicographic. |
| EncOptions.Sort | Description |
|---|---|
| SortNone (default) | No sorting for map keys. |
| SortLengthFirst | Length-first map key ordering. |
| SortBytewiseLexical | Bytewise lexicographic map key ordering (RFC 8949 Section 4.2.1). |
| SortCanonical | (alias) Same as SortLengthFirst (RFC 7049 Section 3.9) |
| SortCTAP2 | (alias) Same as SortBytewiseLexical (CTAP2 Canonical CBOR). |
| SortCoreDeterministic | (alias) Same as SortBytewiseLexical (RFC 8949 Section 4.2.1). |
| EncOptions.Time | Description |
|---|---|
| TimeUnix (default) | (seconds) Encode as integer. |
| TimeUnixMicro | (microseconds) Encode as floating-point. ShortestFloat option determines size. |
| TimeUnixDynamic | (seconds or microseconds) Encode as integer if time doesn't have fractional seconds, otherwise encode as floating-point rounded to microseconds. |
| TimeRFC3339 | (seconds) Encode as RFC 3339 formatted string. |
| TimeRFC3339Nano | (nanoseconds) Encode as RFC3339 formatted string. |
| EncOptions.TimeTag | Description |
|---|---|
| EncTagNone (default) | Tag number will not be encoded for time values. |
| EncTagRequired | Tag number (0 or 1) will be encoded unless time value is undefined/zero-instant. |
By default, undefined (zero instant) time values will encode as CBOR Null without tag number for both EncTagNone and EncTagRequired. Although CBOR Undefined might be technically more correct for EncTagRequired, CBOR Undefined might not be supported by other generic decoders and it isn't supported by JSON.
Go's time package provides IsZero function, which reports whether t represents the zero time instant, January 1, year 1, 00:00:00 UTC.
| EncOptions.BigIntConvert | Description |
|---|---|
| BigIntConvertShortest (default) | Encode big.Int as CBOR integer if value fits. |
| BigIntConvertNone | Encode big.Int as CBOR bignum (tag 2 or 3). |
Floating-Point Options
Encoder has 3 types of options for floating-point data: ShortestFloatMode, InfConvertMode, and NaNConvertMode.
| EncOptions.ShortestFloat | Description |
|---|---|
| ShortestFloatNone (default) | No size conversion. Encode float32 and float64 to CBOR floating-point of same bit-size. |
| ShortestFloat16 | Encode float64 -> float32 -> float16 (IEEE 754 binary16) when values fit. |
Conversions for infinity and NaN use InfConvert and NaNConvert settings.
| EncOptions.InfConvert | Description |
|---|---|
| InfConvertFloat16 (default) | Convert +- infinity to float16 since they always preserve value (recommended) |
| InfConvertNone | Don't convert +- infinity to other representations -- used by CTAP2 Canonical CBOR |
| EncOptions.NaNConvert | Description |
|---|---|
| NaNConvert7e00 (default) | Encode to 0xf97e00 (CBOR float16 = 0x7e00) -- used by RFC 8949 Preferred Encoding, etc. |
| NaNConvertNone | Don't convert NaN to other representations -- used by CTAP2 Canonical CBOR. |
| NaNConvertQuiet | Force quiet bit = 1 and use shortest form that preserves NaN payload. |
| NaNConvertPreserveSignal | Convert to smallest form that preserves value (quit bit unmodified and NaN payload preserved). |
| EncOptions.IndefLength | Description |
|---|---|
| IndefLengthAllowed (default) | allow indefinite length data |
| IndefLengthForbidden | forbid indefinite length data |
| EncOptions.TagsMd | Description |
|---|---|
| TagsAllowed (default) | allow CBOR tags (major type 6) |
| TagsForbidden | forbid CBOR tags (major type 6) |
TagOptions specifies how encoder and decoder handle tag number registered with TagSet.
| TagOptions.DecTag | Description |
|---|---|
| DecTagIgnored (default) | Tag numbers are ignored (if present). |
| DecTagOptional | Tag numbers are only checked for validity if present. |
| DecTagRequired | Tag numbers must be provided except for CBOR Null and CBOR Undefined. |
| TagOptions.EncTag | Description |
|---|---|
| EncTagNone (default) | Tag number will not be encoded. |
| EncTagRequired | Tag number will be encoded. |
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
🛡️ Use Go's io.LimitReader to limit size when decoding very large or indefinite size data.
Functions with identical signatures to encoding/json include:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode.
Default Mode
If default options are acceptable, package level functions can be used for encoding and decoding.
b, err := cbor.Marshal(v) // encode v to []byte b
err := cbor.Unmarshal(b, &v) // decode []byte b to v
encoder := cbor.NewEncoder(w) // create encoder with io.Writer w
decoder := cbor.NewDecoder(r) // create decoder with io.Reader r
Modes
If you need to use options or CBOR tags, then you'll want to create a mode.
"Mode" means defined way of encoding or decoding -- it links the standard API to your CBOR options and CBOR tags. This way, you don't pass around options and the API remains identical to encoding/json.
EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.
For example, em, err := cbor.EncOptions{...}.EncMode() or em, err := cbor.CanonicalEncOptions().EncMode().
EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are reusable, safe for concurrent use, and allow fast parallelism.
Creating and Using Encoding Modes
EncMode is an interface (API) created from EncOptions struct. EncMode uses immutable options after being created and is safe for concurrent use. For best performance, EncMode should be reused.
// Create EncOptions using either struct literal or a function.
opts := cbor.CanonicalEncOptions()
// If needed, modify opts. For example: opts.Time = cbor.TimeUnix
// Create reusable EncMode interface with immutable options, safe for concurrent use.
em, err := opts.EncMode()
// Use EncMode like encoding/json, with same function signatures.
b, err := em.Marshal(v) // encode v to []byte b
encoder := em.NewEncoder(w) // create encoder with io.Writer w
err := encoder.Encode(v) // encode v to io.Writer w
Struct Tags (keyasint, toarray, omitempty)
The keyasint, toarray, and omitempty struct tags make it easy to use compact CBOR message formats. Internet standards often use CBOR arrays and CBOR maps with int keys to save space.
Decoding CWT (CBOR Web Token) using keyasint and toarray struct tags:
// Signed CWT is defined in RFC 8392
type signedCWT struct {
_ struct{} `cbor:",toarray"`
Protected []byte
Unprotected coseHeader
Payload []byte
Signature []byte
}
// Part of COSE header definition
type coseHeader struct {
Alg int `cbor:"1,keyasint,omitempty"`
Kid []byte `cbor:"4,keyasint,omitempty"`
IV []byte `cbor:"5,keyasint,omitempty"`
}
// data is []byte containing signed CWT
var v signedCWT
if err := cbor.Unmarshal(data, &v); err != nil {
return err
}
Encoding CWT (CBOR Web Token) using keyasint and toarray struct tags:
// Use signedCWT struct defined in "Decoding CWT" example.
var v signedCWT
...
if data, err := cbor.Marshal(v); err != nil {
return err
}
Encoding and Decoding CWT (CBOR Web Token) with CBOR Tags
// Use signedCWT struct defined in "Decoding CWT" example.
// Create TagSet (safe for concurrency).
tags := cbor.NewTagSet()
// Register tag COSE_Sign1 18 with signedCWT type.
tags.Add(
cbor.TagOptions{EncTag: cbor.EncTagRequired, DecTag: cbor.DecTagRequired},
reflect.TypeOf(signedCWT{}),
18)
// Create DecMode with immutable tags.
dm, _ := cbor.DecOptions{}.DecModeWithTags(tags)
// Unmarshal to signedCWT with tag support.
var v signedCWT
if err := dm.Unmarshal(data, &v); err != nil {
return err
}
// Create EncMode with immutable tags.
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
// Marshal signedCWT with tag number.
if data, err := cbor.Marshal(v); err != nil {
return err
}
For more examples, see examples_test.go.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
Comparisons are between this newer library and a well-known library that had 1,000+ stars before this library was created. Default build settings for each library were used for all comparisons.
This library is safer. Small malicious CBOR messages are rejected quickly before they exhaust system resources.
Decoding 9 or 10 bytes of malformed CBOR data shouldn't exhaust memory. For example,
[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}
| Decode bad 10 bytes to interface{} | Decode bad 10 bytes to []byte | |
|---|---|---|
| fxamacker/cbor 1.0-2.3 | 49.44 ns/op, 24 B/op, 2 allocs/op* | 51.93 ns/op, 32 B/op, 2 allocs/op* |
| ugorji/go 1.2.6 | ⚠️ 45021 ns/op, 262852 B/op, 7 allocs/op | 💥 runtime: out of memory: cannot allocate |
| ugorji/go 1.1.0-1.1.7 | 💥 runtime: out of memory: cannot allocate | 💥 runtime: out of memory: cannot allocate |
*Speed and memory are for latest codec version listed in the row (compiled with Go 1.17.5).
fxamacker/cbor CBOR safety settings include: MaxNestedLevels, MaxArrayElements, MaxMapPairs, and IndefLength.
This library is smaller. Programs like senmlCat can be 4 MB smaller by switching to this library. Programs using more complex CBOR data types can be 9.2 MB smaller.
This library is faster for encoding and decoding CBOR Web Token (CWT). However, speed is only one factor and it can vary depending on data types and sizes. Unlike the other library, this one doesn't use Go's unsafe package or code gen.
This library uses less memory for encoding and decoding CBOR Web Token (CWT) using test data from RFC 8392 A.1.
| fxamacker/cbor 2.3 | ugorji/go 1.2.6 | |
|---|---|---|
| Encode CWT | 0.18 kB/op 2 allocs/op | 1.35 kB/op 4 allocs/op |
| Decode CWT | 160 bytes/op 6 allocs/op | 744 bytes/op 6 allocs/op |
Running your own benchmarks is highly recommended. Use your most common data structures and data sizes.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
Go structs are faster than maps with string keys:
Go structs with keyasint struct tag are faster than maps with integer keys:
Go structs with toarray struct tag are faster than slice:
Doing your own benchmarks is highly recommended. Use your most common message sizes and data types.
See Benchmarks for fxamacker/cbor.
Over 375 tests must pass on 4 architectures before tagging a release. They include all RFC 7049 and RFC 8949 examples, bugs found by fuzzing, maliciously crafted CBOR data, and over 87 tests with malformed data. There's some overlap in the tests but it isn't a high priority to trim tests.
Code coverage must not fall below 95% when tagging a release. Code coverage is above 98% (go test -cover) for cbor v2.3 which is among the highest for libraries (in Go) of this type.
Coverage-guided fuzzing must pass 1+ billion execs using a large corpus before tagging a release. Fuzzing is usually continued after the release is tagged and is manually stopped after reaching 1-3 billion execs. Fuzzing uses a customized version of dvyukov/go-fuzz.
To prevent delays to release schedules, fuzzing is not restarted for a release if changes are limited to ci, docs, and comments.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
This project uses Semantic Versioning, so the API is always backwards compatible unless the major version number changes.
These functions have signatures identical to encoding/json and they will likely never change even after major new releases:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, and (*Decoder).Decode.
Newly added API documented as "subject to change" are excluded from SemVer.
Newly added API in the master branch that has never been release tagged are excluded from SemVer.
This project has adopted the Contributor Covenant Code of Conduct. Contact faye.github@gmail.com with any questions or comments.
Please refer to How to Contribute.
Security fixes are provided for the latest released version of fxamacker/cbor.
For the full text of the Security Policy, see SECURITY.md.
Phrases like "no crashes", "doesn't crash", and "is secure" mean there are no known crash bugs in the latest version based on results of unit tests and coverage-guided fuzzing. They don't imply the software is 100% bug-free or 100% invulnerable to all known and unknown attacks.
Please read the license for additional disclaimers and terms.
Making this library better
Help clarifying CBOR RFC 7049 or 7049bis (7049bis is the draft of RFC 8949)
Words of encouragement and support
Copyright © 2019-2022 Faye Amacker.
fxamacker/cbor is licensed under the MIT License. See LICENSE for the full license text.
⚓ Quick Start • Features • Standards • API • Options • Usage • Fuzzing • License
FAQs
Package cbor is a modern CBOR codec (RFC 8949 & RFC 7049) with CBOR tags, Go struct tags (toarray/keyasint/omitempty), Core Deterministic Encoding, CTAP2, Canonical CBOR, float64->32->16, and duplicate map key detection. Encoding options allow "preferred serialization" by encoding integers and floats to their smallest forms (e.g. float16) when values fit. Struct tags like "keyasint", "toarray" and "omitempty" make CBOR data smaller and easier to use with structs. For example, "toarray" tag makes struct fields encode to CBOR array elements. And "keyasint" makes a field encode to an element of CBOR map with specified int key. Latest docs can be viewed at https://github.com/fxamacker/cbor#cbor-library-in-go The Quick Start guide is at https://github.com/fxamacker/cbor#quick-start Function signatures identical to encoding/json include: Standard interfaces include: Custom encoding and decoding is possible by implementing standard interfaces for user-defined Go types. Codec functions are available at package-level (using defaults options) or by creating modes from options at runtime. "Mode" in this API means definite way of encoding (EncMode) or decoding (DecMode). EncMode and DecMode interfaces are created from EncOptions or DecOptions structs. Modes use immutable options to avoid side-effects and simplify concurrency. Behavior of modes won't accidentally change at runtime after they're created. Modes are intended to be reused and are safe for concurrent use. EncMode and DecMode Interfaces Using Default Encoding Mode Using Default Decoding Mode Creating and Using Encoding Modes Predefined Encoding Options: https://github.com/fxamacker/cbor#predefined-encoding-options Encoding Options: https://github.com/fxamacker/cbor#encoding-options Decoding Options: https://github.com/fxamacker/cbor#decoding-options Struct tags like `cbor:"name,omitempty"` and `json:"name,omitempty"` work as expected. If both struct tags are specified then `cbor` is used. Struct tags like "keyasint", "toarray", and "omitempty" make it easy to use very compact formats like COSE and CWT (CBOR Web Tokens) with structs. For example, "toarray" makes struct fields encode to array elements. And "keyasint" makes struct fields encode to elements of CBOR map with int keys. https://raw.githubusercontent.com/fxamacker/images/master/cbor/v2.0.0/cbor_easy_api.png Struct tags are listed at https://github.com/fxamacker/cbor#struct-tags-1 Over 375 tests are included in this package. Cover-guided fuzzing is handled by fxamacker/cbor-fuzz.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The June TC39 meeting wrapped up this week with eight proposals moving on to the next stage. Here's a quick roundup of the features that the committee approved to advance.
Security News
Cyber extortion in the US and Canada hit record levels in 2023, with ransomware attacks surging and median ransom demands skyrocketing, though fewer companies are choosing to pay ransoms.
Security News
Ecma TC39 is meeting this week and has moved key ECMAScript proposals forward, advancing Deferred Import Evaluation, Error.isError(), RegExp Escaping, and Promise.try to the next stages.