zstdlib

Zstdlib - a Zstandard data compression library binding for Ruby

zstdlib is native Ruby extension for Zstandard data compression library.

zstdlib is currently available for the MRI Ruby platform.

Why bother?

Unlike the other Zstd bindings available for Ruby, Zstdlib utilizes Zstd's native Zlib compatibility layer.

This specifically means that Zstdlib module is and will be (mostly) API-compatible with standard Ruby Zlib module and thus zstdlib can be used as a drop-in replacement for zlib: just replace Zlib with Zstdlib throughout the source code and you're in!

Streams produced by Zstdlib::Deflate class can be decompressed with standard zstd utility and hence can be written to .zst or .zstd files. Conversely, the Zstdlib::Inflate class can handle data from such files.

Use cases

Simple string compression

require 'zstdlib'
data = Zstdlib.deflate('Hello Zstd')

Incremental data compression

require 'zstdlib'
zstd = Zstdlib::Deflate.new
data = String.new
data << zstd.deflate('Hello')
data << zstd.deflate('Zstd')
data << zstd.finish

Simple string decompression

require 'zstdlib'
data = Zstdlib.inflate(compressed_data)

Technical notes

zstdlib covers the following parts of zlib:

• ZStream, Inflate and Deflate classes
• Various Zlib:: constants (DEFAULT_COMPRESSION etc.)
• Zlib:: exception classes (Error and its descendants)

Note that zstdlib currently omits Gzip file support with its GzipFile, GzipReader and GzipWriter classes and associated constants although this may be changed in future.

There exist Zstdlib::ZSTD_VERSION constant and Zstdlib.zstd_version module method which can be used to obtain shipped Zstd library version.

General guidelines

In order to enable Zstd (de)compression capability in existing code, simply replace zlib with zstdlib in require statements and Zlib with Zstdlib module references throughout the code.

The rest of the zlib-aware code should work unchanged.

For further information refer to documentation on zlib .

Notes on compression levels

If unsure do not pass anything to constructor in order to use DEFAULT_COMPRESSION level which works for both zlib ans zstdlib.

Contrary to zlib the NO_COMPRESSION constant in zstdlib is an equivalent to BEST_SPEED as Zstd always compresses data.

The BEST_COMPRESSION constant is adjusted to reflect Zstd-specific compression level range. Anyways this level is recommended against due to abysmal performance not to be expected from the fast state-of-the-art compression algorithm. The Zstd documentation (currently) recommends using 19 as the level for optimal best compression.

The BEST_SPEED constant remains unchanged.

Availability

zstdlib home page on GitHub.

Source code and system-specific compiled gems can be obtained from rubygems.org.

The system-specific gems are currently provided for x32/x64 Windows and x64/ARM64 MacOS platforms.

Note that these gems are built with the rake-compiler-dock infrastructure the minimum supported Ruby version is raised to 2.4. All other installations including Linux will use the source gem and hence require the compilation step with locally installed development tools.

Release history

For user-visible changes refer to change log.

Caveats

This module is in testing phase so things beyond basic interface might not work as expected.

Documentation extracted from zstdlib still contains traces of old zlib-related pieces. Just keep this in mind and substitute zlib with zstdlib upon browsing.

Gzip .gz file support, although available in Zstd's Zlib compatibility layer, is currently disabled.

Zstd's external compression dictionaries capability is not (yet) implemented.

The end

Cheers,

Oleg A. Khlybov <[e-mail]>