token-types

What is token-types?

The token-types npm package provides a collection of token parsers for binary data types, facilitating the reading and interpretation of various data types from binary streams. This is particularly useful in applications dealing with multimedia files, where metadata and content need to be extracted or interpreted from binary formats.

What are token-types's main functionalities?

Reading integer values

This feature allows the reading of unsigned 32-bit integers from a buffer in little-endian format. It is useful for parsing low-level binary data structures.

const Token = require('token-types');

const uint32Token = Token.UINT32_LE;
const buffer = Buffer.from([0x01, 0x00, 0x00, 0x00]);
const value = uint32Token.get(buffer, 0);
console.log(value); // Outputs: 1

Reading string values

This feature enables the extraction of ASCII strings from binary data. It supports specifying the length of the string and the encoding, making it versatile for reading text data embedded in binary streams.

const Token = require('token-types');

const stringToken = new Token.StringType(5, 'ascii');
const buffer = Buffer.from('hello');
const value = stringToken.get(buffer, 0);
console.log(value); // Outputs: 'hello'

Other packages similar to token-types

binary-parser

binary-parser offers functionality to build a schema for parsing binary data, which is somewhat similar to token-types. However, binary-parser is more comprehensive in defining complex parsing schemas, making it suitable for applications that require detailed and structured binary data interpretation.

README

A primitive token library used to read from, and to write a node Buffer.

Tokens

node-strtok supports a wide variety of numerical tokens out of the box:

• UINT8
• UINT16_BE, UINT16_LE
• UINT24_BE, UINT24_LE
• UINT32_BE, UINT32_LE
• INT8
• INT16_BE, INT16_LE
• INT24_BE, INT24_LE
• INT32_BE, INT32_LE

String types:

• Windows-1252
• ISO-8859-1

One might notice that there is no support for 64-bit tokens, since JavaScript seems to limit value size to less than 2^64. Rather than wrapping up an additional math library to handle this, I wanted to stick with JavaScript primitives. Maybe this will change later if this becomes important.