compress-utils

compress-utils

A unified, high-performance interface for multiple compression algorithms across programming languages.

Features

• Multiple Algorithms: Integrated support for Brotli, bzip2, lz4, LZMA, XZ, zlib, and Zstandard
• Consistent API: Same interface across all supported algorithms
• Cross-Language: Core C++ library with language bindings for Python (and more to come)
• Performance Focused: Built to minimize overhead over native compression libraries

Installation

pip install compress-utils

Quick Start

Object-Oriented API

from compress_utils import compressor, Algorithm

# Create a compressor with your algorithm of choice
comp = compressor("zstd")  # or 'brotli', 'xz', etc.

# Compress data with optional compression level (1-10)
compressed = comp.compress(b"Hello, world!", level=3)

# Decompress data
original = comp.decompress(compressed)

Functional API

from compress_utils import compress, decompress

# Compress and decompress directly
compressed = compress(b"Hello, world!", "zstd", level=3)
original = decompress(compressed, "zstd")

Streaming API

For processing large data in chunks or when data arrives incrementally (e.g., from a network stream), use the streaming API:

from compress_utils import CompressStream, DecompressStream

# Compression streaming
stream = CompressStream("zstd", level=3)
compressed_chunks = []

# Process data in chunks
for chunk in data_chunks:
    compressed_chunks.append(stream.compress(chunk))

# Finalize compression (important!)
compressed_chunks.append(stream.finish())

# Decompression streaming
decompress_stream = DecompressStream("zstd")
decompressed_chunks = []

for chunk in compressed_chunks:
    decompressed_chunks.append(decompress_stream.decompress(chunk))

# Finalize decompression
decompressed_chunks.append(decompress_stream.finish())

The streaming API is ideal for:

• Processing files that don't fit in memory
• Network data that arrives in chunks
• Real-time compression/decompression
• Pipeline processing where data flows through multiple stages

Available Algorithms

The following algorithms are supported (availability depends on build configuration):

• Brotli - Google's compression algorithm optimized for the web
• LZMA/XZ - High compression ratio algorithms
• zlib - Widely used general-purpose compression
• Zstandard - Fast compression algorithm with high ratios

Documentation

For detailed API documentation and advanced usage, see the full documentation.

Contributing

Contributions are welcome! Check out the issues page for open tasks or submit your own ideas.

License

This project is licensed under the MIT License - see the LICENSE file for details.