Compression

The missing Python utility to read and write large compressed JSONs.

Python bindings for Vortex, an Apache Arrow-compatible toolkit for working with compressed array data.

Compress stream by GZIP on the fly.

Flexible Polyline encoding: a lossy compressed representation of a list of coordinate pairs or triples

A Python port of the YUI CSS compression algorithm.

Fluid dynamics component of Chemical Engineering Design Library (ChEDL)

A library for handling common compression formats for N64 games

Infrastructure for support neural networks compression

pathlib.Path adapters to transparently read data from compressed files and folders

Limited Error Raster Compression

A library for compressing large language models utilizing the latest techniques and research in the field for both training aware and post training techniques. The library is designed to be flexible and easy to use on top of PyTorch and HuggingFace Transformers, allowing for quick experimentation.

A nanobind python API for apngasm, a tool/library for APNG assembly & disassembly with compression support.

Python wrapper for multithreaded .png image file optimizer oxipng

Data compression methods and benchmarks for Cli/Met datasets

Data compression in TensorFlow

Deduplicated, encrypted, authenticated and compressed backups

A family of highly efficient, lightweight yet powerful optimizers.

Python bindings for the Xpress9 compression library

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Format and compress XML documents

Good compression for numerical sequences

Crackle 3D dense segmentation compression codec.

Python bindings for the LZMA library by Igor Pavlov.

A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP generation, change opset, change to the specified input order, addition of OP, RGB to BGR conversion, change batch size, batch rename of OP, and JSON convertion for ONNX models.

A comprehensive knowledge distillation training framework for transformer models

SC Compression

Python bindings for the LZO data compression library

A Python Geohash Compression Tool

Neuroglancer compressed_segmentation codec.

Geobuf is a compact binary geospatial format for lossless compression of GeoJSON.

A Python and rust library for encoding and decoding blurhash strings

Tar (and compress) files in s3

Infrastructure for support neural networks compression

Internet Archive PDF compression tools

AI-Optimized Hybrid Compression Protocol for Real-Time Communication

LZW compression and decompression

Compressible Finite Volume Solver for Atmospheric Dynamics, Chemistry and Thermodynamics

A more pythonic wrapper around the MSNumpress library for mass spectrometry numerical data compression

High-performance FM-index powered by Rust, enabling fast substring search and count/locate queries.

Pure Python implementation of the XZ file format with random access support

Columnar and compressed data containers.

Precompiled binaries for pngquant, the lossy PNG compressor based on libimagequant.

Python bindings for a Rust implementation of arithmetic coding (encoder/decoder).

Effortlessly compress / decompress any RINEX file

Adaptive Guarded Object Notation - A schema-driven, token-efficient data interchange format for LLMs

zfp compression in Python

A static files storage backend for compression using GZip and/or Brotli that inherits from Djangos ManifestFilesMixin and StaticFilesStorage; also minifies static files.

Library to compress LSTM layers

Python bindings for Google's Guetzli, a JPEG encoder that optimises JPEG compression