@thi.ng/bitstream

What is @thi.ng/bitstream?

@thi.ng/bitstream is a TypeScript/JavaScript library for working with bit-level data streams. It provides utilities for reading, writing, and manipulating bits in a highly efficient manner. This package is useful for applications that require low-level data processing, such as encoding/decoding, compression, and cryptographic operations.

What are @thi.ng/bitstream's main functionalities?

BitStreamWriter

The BitStreamWriter class allows you to write individual bits or groups of bits to a stream. This is useful for creating custom binary formats or for bit-level data compression.

const { BitStreamWriter } = require('@thi.ng/bitstream');

const writer = new BitStreamWriter();
writer.write(0b101, 3); // write 3 bits
writer.write(0b1111, 4); // write 4 bits
console.log(writer.bytes()); // Uint8Array containing the written bits

BitStreamReader

The BitStreamReader class allows you to read individual bits or groups of bits from a stream. This is useful for parsing custom binary formats or for bit-level data decompression.

const { BitStreamReader } = require('@thi.ng/bitstream');

const reader = new BitStreamReader(new Uint8Array([0b10111100]));
console.log(reader.read(3)); // read 3 bits, output: 5 (0b101)
console.log(reader.read(4)); // read 4 bits, output: 15 (0b1111)

Bit manipulation utilities

The package provides various utility functions for manipulating individual bits within a byte or larger data structure. These functions include setting, clearing, toggling, and getting the value of specific bits.

const { setBit, clearBit, toggleBit, getBit } = require('@thi.ng/bitstream');

let byte = 0b00001111;
byte = setBit(byte, 7); // set the 7th bit, byte becomes 0b10001111
byte = clearBit(byte, 3); // clear the 3rd bit, byte becomes 0b10000111
byte = toggleBit(byte, 0); // toggle the 0th bit, byte becomes 0b10000110
console.log(getBit(byte, 7)); // get the 7th bit, output: 1

Other packages similar to @thi.ng/bitstream

bit-buffer

bit-buffer is a library for reading and writing bits to a buffer. It provides similar functionality to @thi.ng/bitstream, including bit-level reading and writing, but it is more focused on working with buffers rather than streams.

bitwise

bitwise is a library that provides a collection of bitwise operations for JavaScript. It includes utilities for bit manipulation, similar to @thi.ng/bitstream, but it does not provide stream-based reading and writing capabilities.

bitjs

bitjs is a JavaScript library for bit-level operations, including reading and writing bits, as well as bit manipulation. It offers similar functionality to @thi.ng/bitstream but is less comprehensive in terms of stream handling.

README

[!NOTE] This is one of 189 standalone projects, maintained as part of the @thi.ng/umbrella monorepo and anti-framework.

🚀 Please help me to work full-time on these projects by sponsoring me on GitHub. Thank you! ❤️

• About
• Status
• Related packages
• Installation
• Dependencies
• API
  • BitOutputStream
  • BitInputStream
  • Barebones alternatives
• Authors
• License

About

ES6 iterator based read/write bit streams with support for variable word widths.

Status

STABLE - used in production

Search or submit any issues for this package

Related packages

• @thi.ng/range-coder - Binary data range encoder / decoder
• @thi.ng/rle-pack - Binary run-length encoding packer w/ flexible repeat bit widths

Installation

yarn add @thi.ng/bitstream

ESM import:

import * as bit from "@thi.ng/bitstream";

Browser ESM import:

<script type="module" src="https://esm.run/@thi.ng/bitstream"></script>

JSDelivr documentation

For Node.js REPL:

const bit = await import("@thi.ng/bitstream");

Package sizes (brotli'd, pre-treeshake): ESM: 1.33 KB

Dependencies

• @thi.ng/errors

API

Generated API docs

BitOutputStream

Uint8Array backed, bitwise output stream abstraction (big endian order). Individual word sizes can range between 1-52 bits (in practice) and are not fixed (each word can have a different size).

The constructor accepts an optional initial Uint8Array buffer or buffer size (in bytes) and an optional write start position (in bits). The buffer will only be written to starting from the given bit position (even if in the middle of a byte). Default buffer size is 16 bytes, but the array is resized (x2) automatically each time capacity is reached.

Note: The max. word size of 52 bits is not enforced by the library, but JS can only represent integers (w/o loss of precision) up to 2^53-1. If you're willing to accept lossy precision for larger values, technically the max. supported word width is 64 bits.

import { BitOutputStream } from "@thi.ng/bitstream";

out = new BitOutputStream();
// write 3-bit number (only the lowest 3 bits are used, here 0x05)
out.write(0xf5, 3);
// write 7-bit number
out.write(0x66, 7);
// write 32-bit number
out.write(0xdecafbad, 32);

// write values from given iterable (w/ fixed word size, here 16 bits)
out.writeWords([0xaaaa, 0x5555], 16);

// get bytes (only up to current write position)
out.bytes()
// Uint8Array [ 185, 183, 178, 190, 235, 106, 170, 149, 85, 64 ]

In addition to the generic write() method, there's also the slightly faster writeBit() for writing single bits (the arg MUST be 0 or 1 only).

Using seek(pos), the write position can be repositioned within current limits (does not attempt to resize backing buffer).

BitInputStream

Uint8Array backed bitwise input stream abstraction (big endian order) with optional start position and read limit (both in bits). All readers are independent instances, but if obtained from BitOutputStream will share the same backing buffer as the writer. An auto-configured input stream can also be obtained via output.reader(). The class too implements the ES6 Iterator API for bitwise read access, as well as a read() method to read bitfields.

Note: Attempting to read beyond capacity will throw an EOF error.

Using input.seek(pos), the read position can be repositioned within stream limits.

// get input from output stream (see above)...
// (for reference, the ^ indicate the start of each bit field)
[...out.reader()].join("")
// "10111001101101111011001010111110111010110110101010101010100101010101010101"
//  ^  ^      ^                               ^               ^

// obtain new reader
input = out.reader();
// sequentially read bit fields of varying sizes
out.reader().readFields([3, 7, 32, 16, 16]).map(x => x.toString(16))
// [ "5", "66", "decafbad", "aaaa", "5555" ]

// or read fields into object
out.reader().readStruct([["a", 3], ["b", 7], ["c", 32], ["d", 16], ["e", 16]]);
// { a: 5, b: 102, c: 3737844653, d: 43690, e: 21845 }

// or read a number of fixed size words (here also from given pos)
out.reader().seek(10).readWords(4, 16).map(x=>x.toString(16));
// [ 'deca', 'fbad', 'aaaa', '5555' ]

import { BitInputStream } from "@thi.ng/bitstream";

src = new Uint8Array([0xf1,0xe2,0xd3,0xc4,0xb5,0xa6,0x97,0x88]);
// create stream from bit 36
input = new BitInputStream(src, 36);
input.read(12).toString(16);
// 5a6
input.read(4)
// 9
input.read(4)
// 7
input.read(1) // or use input.readBit()
// 1 => msb of last byte (0x88)
input.read(7)
// 8 => low nibble of last byte

In addition to the generic read() method, there's also the slightly faster readBit() for reading single bits.

Barebones alternatives

For use cases requiring only word sizes <=8 bits and none of the advanced features provided by the above implementations, the package also provides functional barebones alternatives in the form of bitWriter() and bitReader():

import { bitReader, bitWriter } from "@thi.ng/bistream";

const writer = bitWriter();
// write single bit
writer.write(1);

// write unsigned value (up to 8 bits)
writer.write(31, 5);

// retrieve buffer
const bytes = writer.bytes();
// Uint8Array(1) [ 252 ]

// create reader from byte buffer
const reader = bitReader(bytes);

// read single bit
reader();
// 1

// read n-bit unsigned value
reader(5);
// 31

Authors

• Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

@misc{thing-bitstream,
  title = "@thi.ng/bitstream",
  author = "Karsten Schmidt",
  note = "https://thi.ng/bitstream",
  year = 2016
}

License

© 2016 - 2024 Karsten Schmidt // Apache License 2.0