cramjam

cramjam

API Documentation

Install

pip install --upgrade cramjam  # Requires no Python or system dependencies!

CLI

A CLI interface is available as cramjam-cli

libcramjam

A Rust crate and C friendly library available at libcramjam

Extremely thin and easy-to-install Python bindings to de/compression algorithms in Rust. Allows for using algorithms such as Snappy, without any system or other python dependencies.

Benchmarks

Some basic benchmarks are available in the benchmarks directory

Available algorithms:

• Snappy      cramjam.snappy
• Brotli          cramjam.brotli
• Bzip2          cramjam.bzip2
• Lz4              cramjam.lz4
• Gzip            cramjam.gzip
• Zlib              cramjam.zlib
• Deflate       cramjam.deflate
• ZSTD           cramjam.zstd
• XZ / LZMA  cramjam.xz
• Blosc2         cramjam.experimental.blosc2
• ISA-L backend (only on 64-bit targets)
  • igzip          cramjam.experimental.igzip
  • ideflate     cramjam.experimental.ideflate
  • izlib           cramjam.experimental.izlib

All available for use as:

>>> import cramjam
>>> import numpy as np
>>> compressed = cramjam.snappy.compress(b"bytes here")
>>> decompressed = cramjam.snappy.decompress(compressed)
>>> decompressed
cramjam.Buffer(len=10)  # an object which implements the buffer protocol
>>> bytes(decompressed)
b"bytes here"
>>> np.frombuffer(decompressed, dtype=np.uint8)
array([ 98, 121, 116, 101, 115,  32, 104, 101, 114, 101], dtype=uint8)

Where the API is cramjam.<compression-variant>.compress/decompress and accepts bytes/bytearray/numpy.array/cramjam.File/cramjam.Buffer / memoryview objects.

de/compress_into Additionally, all variants support decompress_into and compress_into. Ex.

>>> import numpy as np
>>> from cramjam import snappy, Buffer
>>>
>>> data = np.frombuffer(b'some bytes here', dtype=np.uint8)
>>> data
array([115, 111, 109, 101,  32,  98, 121, 116, 101, 115,  32, 104, 101,
       114, 101], dtype=uint8)
>>>
>>> compressed = Buffer()
>>> snappy.compress_into(data, compressed)
33  # 33 bytes written to compressed buffer
>>>
>>> compressed.tell()  # Where is the buffer position?
33  # goodie!
>>>
>>> compressed.seek(0)  # Go back to the start of the buffer so we can prepare to decompress
>>> decompressed = b'0' * len(data)  # let's write to `bytes` as output
>>> decompressed
b'000000000000000'
>>>
>>> snappy.decompress_into(compressed, decompressed)
15  # 15 bytes written to decompressed
>>> decompressed
b'some bytes here'