byte-data

byte-data

JavaScript binary parser for any browser or environment.
Copyright (c) 2017-2019 Rafael da Silva Rocha.
https://github.com/rochars/byte-data

byte-data is JavaScript binary parser for any browser or environment.

• MIT licensed
• Compatible with IE6+ and any environment with ES3/ES5/ES6+ support
• Tested in little-endian and big-endian machines!

Pack/unpack:

• Integers, unsigned and signed (two's complement)
• 16-bit half-precision floating-point numbers
• 32-bit single-precision floating-point numbers
• 64-bit double-precision floating-point numbers
• Little-endian and big-endian words
• UTF-8 strings (1 to 4 bytes per character, invalid characters are replaced)

Install

npm install byte-data

In the Browser

Use the byte-data.js file in the /dist folder:

<script src="./dist/byte-data.js"></script>
<script>
  // Pack a 32-bit floating-point number
  var packed = byteData.pack(2.1474836, {bits: 32, fp: true});
</script>

Or load it from the jsDelivr CDN:

<script src="https://cdn.jsdelivr.net/npm/byte-data"></script>
<script>
  var packed = byteData.pack(2.1474836, {bits: 32, fp: true});
</script>

Or load it from unpkg:

<script src="https://unpkg.com/byte-data"></script>
<script>
  var packed = byteData.pack(2.1474836, {bits: 32, fp: true});
</script>

Browser compatibility

This module is distributed as a minified UMD transpiled to ES3 and compatible with IE6+. It should work in all modern browsers and environments that support ES3/ES5/ES6+. It is located in the dist/ folder.

The UMD is bundled with some polyfills so no extra work should be necessary to load this module in any browser. The polyfills used in the compilation are distributed with the package in the scripts/ folder.

If you are not using a package manager to install this module, you can get the it via CDNs:

<script src="https://cdn.jsdelivr.net/npm/byte-data"></script>
<script src="https://unpkg.com/byte-data"></script>

Cross-browser tests powered by

Node

const byteData = require('byte-data');

// Pack a signed 16-bit integer to a existing byte buffer
// Start writing on index '4' of the buffer
byteData.packTo(1077, {bits: 16, signed: true}, buffer, 4);

// Pack a usigned 8-bit unsigned integer, returns a
// array with the number represented as bytes
let packed = byteData.pack(128, {bits: 8});

Or import:

import * as byteData from 'byte-data';

// Pack a 32-bit floating-point number
let packed = byteData.pack(2.1474836, {bits: 32, fp: true});

Or import just what you need:

import { pack } from 'byte-data';

// Pack a 8-bit unsigned integer
let packed = pack(128, {bits: 8});

About

pack and packTo

pack(num, theType) will return a Array with the bytes of the passed value.

let packed = pack(123, {bits: 16});

packTo(num, theType, buffer, index) will write the bytes of the number to the provided buffer (Uint8Array or Array), start writing on index.

let buffer = new Uint8Array(4);
packTo(402, {bits: 16}, buffer, 2);

index can be ommited and will default to zero:

let buffer = new Uint8Array(4);
packTo(402, {bits: 16}, buffer);

Packing null, false, true and undefined

Packing the following values

• undefined
• null
• true
• false

will values throw a TypeError.

Unpacking and input buffer length

unpack(buffer, theType, index=)

When unpacking a single value, a 'Bad buffer length' error is raised if the number of bytes is not sufficient (Ex: unpack a 32-bit number, but provide a input buffer with length smaller than 4) or if there are not enough bytes after the index.

// throws a 'Bad buffer length' error
byteData.unpack([0xff], {bits: 16}, 0);

// throws a 'Bad buffer length' error
byteData.unpack([0xff, 0xff, 0xff], {bits: 16}, 2);

// ok
byteData.unpack([0xff, 0xff, 0xff], {bits: 16}, 1); 

unpackArray(buffer, theType, start=, end=, safe=)

When unpacking a array of values, extra bytes in the end of the buffer are ignored and insufficient bytes will return a empty array by default:

// return a empty array
byteData.unpackArray([0xff], {bits: 16}, 0, 1);

// return a array with one 16-bit unsigned int
byteData.unpackArray([0xff, 0xff, 0xff], {bits: 16}, 0, 2);

You can unpack arrays in safe mode with the optional safe param set to true. In safe mode insufficient bytes in the input array or extra bytes in the end of the input array will cause a 'Bad buffer length' error:

// throws a 'Bad buffer length' error
byteData.unpackArray([0xff], {bits: 16}, 0, 1, true);

// throws a 'Bad buffer length' error
byteData.unpackArray([0xff, 0xff, 0xff], {bits: 16}, 0, 3, true);

Floating-point numbers

• Floating-point numbers are IEEE 754 standard.
• Overflows are rounded towards Infinity and -Infinity.
• NaN is packed as quiet NaN. Both quiet NaN and signaling NaN can be unpacked.
• Support packing and unpacking negative zeros.
• Support packing and unpacking Infinity and negative Infinity

Minifloats

Currently only 16-bit half-precision.

Integers

• Overflow on integers will throw a RangeError.
• packing NaN will throw a TypeError.
• packing Infinity or -Infinity will throw a RangeError.

To clamp integers on overflow and avoid RangeError, set the optional clamp param to true:

// Set clamp to true; values will be packed
// as their max and min values on overflow
pack(value, theType, true);
packTo(value, theType, buffer, index, true);
packArrayTo(values, theType, buffer, index, true);
packArray(values, theType, true);

Signed integers

Signed integers are two's complement.

Strings

UTF-8 strings with 1 to 4 bytes per character can be packed and unpacked. BOM is kept untouched if present. Invalid characters are replaced with Unicode Character 'REPLACEMENT CHARACTER' (U+FFFD).

Reading strings from buffers

Use unpackString(buffer, index, end). The paramters index and end determine a slice of the buffer to read. End is non-inclusive. So to read the first 4 bytes of a buffer:

let str = unpackString(buffer, 0, 4);
// read from buffer[0], buffer[1], buffer[2], buffer[3]

If index and end are ommited unpackString(buffer) will read the entire buffer:

let str = unpackString(buffer);

Writing strings to buffers

packStringTo(str, buffer, index=0) will write the string to the provided buffer (Uint8Array or Array), starting on the index. Index defaults to zero if ommited (start from the beginning of the buffer).

let buffer = [];
packStringTo(str, buffer);

Packing strings to Uint8Array buffers

If you need to calculate the buffer length before writing you may use the utf8-buffer-size module:

import utf8BufferSize from 'utf8-buffer-size';
let buffer = new Uint8Array(utf8BufferSize(str));
packStringTo(str, buffer);

Types

Types are user-defined objects like this:

const binary32 = {
  bits: 32, // required
  signed: true, // optional, defaults to false
  fp: true, // optional, defaults to false, true for floating-point numbers
  be: false // optional, defaults to false, true for big-endian
}

Tests on big-endian systems

Use QEMU with this PowerPC/Debian image:
https://people.debian.org/~aurel32/qemu/powerpc/

API

// Strings
/**
 * Read a string of UTF-8 characters from a byte buffer.
 * @param {!(Uint8Array|Array<number>)} buffer A byte buffer.
 * @param {number} [index=0] The buffer index to start reading.
 * @param {number} [end=buffer.length] The index to stop reading, non inclusive.
 * @return {string}
 */
function unpackString(buffer, index=0, len=buffer.length) {}

/**
 * Write a string of UTF-8 characters as a byte buffer.
 * @param {string} str The string to pack.
 * @return {!Array<number>} The UTF-8 string bytes.
 */
function packString(str) {}

/**
 * Write a string of UTF-8 characters to a byte buffer.
 * @param {string} str The string to pack.
 * @param {!(Uint8Array|Array<number>)} buffer The output buffer.
 * @param {number} [index=0] The buffer index to start writing.
 * @return {number} The next index to write in the buffer.
 */
function packStringTo(str, buffer, index=0) {}

// Numbers
/**
 * Pack a array of numbers to a byte buffer.
 * All other packing functions are interfaces to this function.
 * @param {!(Array<number>|TypedArray)} values The values to pack.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {!(Uint8Array|Array<number>)} buffer The buffer to write on.
 * @param {number} [index=0] The buffer index to start writing.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {number} The next index to write.
 * @throws {Error} If the type definition is not valid.
 * @throws {RangeError} On overflow.
 * @throws {TypeError} If input is not valid.
 */
function packArrayTo(values, theType, buffer, index=0, clamp=false) {}

/**
 * Unpack a array of numbers from a byte buffer to a array or a typed array.
 * All other unpacking functions are interfaces to this function.
 * @param {!(Uint8Array|Array<number>)} buffer The byte buffer.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {!(TypedArray|Array<number>)} output The output array or typed array.
 * @param {number} [start=0] The buffer index to start reading.
 * @param {number} [end=buffer.length] The buffer index to stop reading.
 * @param {boolean} [safe=false] If set to false, extra bytes in the end of
 *   the array are ignored and input buffers with insufficient bytes will
 *   write nothing to the output array. If safe is set to true the function
 *   will throw a 'Bad buffer length' error.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @throws {Error} If the type definition is not valid
 * @throws {RangeError} On overflow
 */
function unpackArrayTo(
  buffer, theType, output, index=0, end=buffer.length,
  safe=false, clamp=false) {}

/**
 * Pack a number to a byte buffer.
 * @param {number} value The value.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {!(Uint8Array|Array<number>)} buffer The byte buffer to write on.
 * @param {number} [index=0] The buffer index to write.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {number} The next index to write.
 * @throws {Error} If the type definition is not valid.
 * @throws {RangeError} On overflow.
 * @throws {TypeError} If input is not valid.
 */
function packTo(value, theType, buffer, index=0, clamp=false) {}

/**
 * Pack a number as a array of bytes.
 * @param {number} value The number to pack.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {!Array<number>} The packed value.
 * @throws {Error} If the type definition is not valid.
 * @throws {RangeError} On overflow.
 * @throws {TypeError} If input is not valid.
 */
function pack(value, theType, clamp=false) {}

/**
 * Pack a array of numbers as a array of bytes.
 * @param {!(Array<number>|TypedArray)} values The values to pack.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {!Array<number>} The packed values.
 * @throws {Error} If the type definition is not valid.
 * @throws {RangeError} On overflow.
 * @throws {TypeError} If input is not valid.
 */
function packArray(values, theType, clamp=false) {}

/**
 * Unpack a array of numbers from a byte buffer.
 * @param {!(Uint8Array|Array<number>)} buffer The byte buffer.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {number} [start=0] The buffer index to start reading.
 * @param {number} [end=buffer.length] The buffer index to stop reading.
 * @param {boolean} [safe=false] If set to false, extra bytes in the end of
 *   the array are ignored and input buffers with insufficient bytes will
 *   output a empty array. If safe is set to true the function
 *   will throw a 'Bad buffer length' error.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {!Array<number>}
 * @throws {Error} If the type definition is not valid
 * @throws {RangeError} On overflow
 */
function unpackArray(
  buffer, theType, start=0, end=buffer.length, safe=false, clamp=false) {}

/**
 * Unpack a number from a byte buffer.
 * @param {!(Uint8Array|Array<number>)} buffer The byte buffer.
 * @param {!{bits:number,
 *   fp: (boolean|undefined),
 *   signed: (boolean|undefined),
 *   be: (boolean|undefined)}} theType The type definition.
 * @param {number} [index=0] The buffer index to read.
 * @param {boolean} [clamp=false] True to clamp ints on overflow.
 * @return {number}
 * @throws {Error} If the type definition is not valid
 * @throws {Error} On bad buffer length.
 * @throws {RangeError} On overflow
 */
function unpack(buffer, theType, index=0, clamp=false) {}

Contributing

byte-data welcomes all contributions from anyone willing to work in good faith with other contributors and the community. No contribution is too small and all contributions are valued.

See CONTRIBUTING.md for details.

Style guide

byte-data code should follow the Google JavaScript Style Guide:
https://google.github.io/styleguide/jsguide.html

Code of conduct

This project is bound by a code of conduct: The Contributor Covenant, version 1.4, also available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

Reporting issues

Use the GitHub issue tracker.

Reporting security issues

Report security issues to this e-mail: [e-mail].

Legal

LICENSE

Copyright (c) 2017-2019 Rafael da Silva Rocha.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.