Product
Introducing License Enforcement in Socket
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
github.com/zeebo/blake3
Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration.
Special thanks to the excellent avo making writing vectorized version much easier.
This library makes some different design decisions than the upstream Rust crate around internal buffering. Specifically, because it does not target the embedded system space, nor does it support multithreading, it elects to do its own internal buffering. This means that a user does not have to worry about providing large enough buffers to get the best possible performance, but it does worse on smaller input sizes. So some notes:
In this case, both libraries are able to avoid a lot of data copying and will use vectorized instructions to hash as fast as possible, and perform similarly.
For incremental writes, you must provide the Rust version large enough buffers so that it can use vectorized instructions. This Go library performs consistently regardless of the size being sent into the update function.
The downside of internal buffering is most apparent with small sizes as most time is spent initializing the hasher state. In terms of hashing rate, the difference is 3-4x, but in an absolute sense it's ~100ns (see tables below). If you wish to hash a large number of very small strings and you care about those nanoseconds, be sure to use the Reset method to avoid re-initializing the state.
Size | Full Buffer | Reset | Full Buffer Rate | Reset Rate | |
---|---|---|---|---|---|
64 b | 205ns | 86.5ns | 312MB/s | 740MB/s | |
256 b | 364ns | 250ns | 703MB/s | 1.03GB/s | |
512 b | 575ns | 468ns | 892MB/s | 1.10GB/s | |
768 b | 795ns | 682ns | 967MB/s | 1.13GB/s |
Size | Incremental | Full Buffer | Reset | Incremental Rate | Full Buffer Rate | Reset Rate | |
---|---|---|---|---|---|---|---|
1 kib | 1.02µs | 1.01µs | 891ns | 1.00GB/s | 1.01GB/s | 1.15GB/s | |
2 kib | 2.11µs | 2.07µs | 1.95µs | 968MB/s | 990MB/s | 1.05GB/s | |
4 kib | 2.28µs | 2.15µs | 2.05µs | 1.80GB/s | 1.90GB/s | 2.00GB/s | |
8 kib | 2.64µs | 2.52µs | 2.44µs | 3.11GB/s | 3.25GB/s | 3.36GB/s | |
16 kib | 4.93µs | 4.54µs | 4.48µs | 3.33GB/s | 3.61GB/s | 3.66GB/s | |
32 kib | 9.41µs | 8.62µs | 8.54µs | 3.48GB/s | 3.80GB/s | 3.84GB/s | |
64 kib | 18.2µs | 16.7µs | 16.6µs | 3.59GB/s | 3.91GB/s | 3.94GB/s | |
128 kib | 36.3µs | 32.9µs | 33.1µs | 3.61GB/s | 3.99GB/s | 3.96GB/s | |
256 kib | 72.5µs | 65.7µs | 66.0µs | 3.62GB/s | 3.99GB/s | 3.97GB/s | |
512 kib | 145µs | 131µs | 132µs | 3.60GB/s | 4.00GB/s | 3.97GB/s | |
1024 kib | 290µs | 262µs | 262µs | 3.62GB/s | 4.00GB/s | 4.00GB/s |
Size | Incremental | Full Buffer | Reset | Incremental Rate | Full Buffer Rate | Reset Rate | |
---|---|---|---|---|---|---|---|
64 b | 253ns | 254ns | 134ns | 253MB/s | 252MB/s | 478MB/s | |
256 b | 553ns | 557ns | 441ns | 463MB/s | 459MB/s | 580MB/s | |
512 b | 948ns | 953ns | 841ns | 540MB/s | 538MB/s | 609MB/s | |
768 b | 1.38µs | 1.40µs | 1.35µs | 558MB/s | 547MB/s | 570MB/s | |
1 kib | 1.77µs | 1.77µs | 1.70µs | 577MB/s | 580MB/s | 602MB/s | |
1024 kib | 880µs | 883µs | 878µs | 596MB/s | 595MB/s | 598MB/s |
The speed caps out at around 1 kib, so most rows have been elided from the presentation.
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
Product
We're launching a new set of license analysis and compliance features for analyzing, managing, and complying with licenses across a range of supported languages and ecosystems.
Product
We're excited to introduce Socket Optimize, a powerful CLI command to secure open source dependencies with tested, optimized package overrides.