mmh3

mmh3
is a Python extension for
MurmurHash (MurmurHash3), a set of
fast and robust non-cryptographic hash functions invented by Austin Appleby.
By combining mmh3
with probabilistic techniques like
Bloom filter,
MinHash, and
feature hashing, you can
develop high-performance systems in fields such as data mining, machine
learning, and natural language processing.
Another popular use of mmh3
is to
calculate favicon hashes,
which are utilized by Shodan, the world's first IoT
search engine.
This page provides a quick start guide. For more comprehensive information,
please refer to the documentation.
Installation
pip install mmh3
Usage
Basic usage
>>> import mmh3
>>> mmh3.hash(b"foo")
-156908512
>>> mmh3.hash("foo")
-156908512
>>> mmh3.hash(b"foo", 42)
-1322301282
>>> mmh3.hash(b"foo", 0, False)
4138058784
mmh3.mmh3_x64_128_digest()
, introduced in version 5.0.0, efficienlty hashes
buffer objects that implement the buffer protocol
(PEP 688) without internal memory copying.
The function returns a bytes
object of 16 bytes (128 bits). It is
particularly suited for hashing large memory views, such as
bytearray
, memoryview
, and numpy.ndarray
, and performs faster than
the 32-bit variants like hash()
on 64-bit machines.
>>> mmh3.mmh3_x64_128_digest(numpy.random.rand(100))
b'\x8c\xee\xc6z\xa9\xfeR\xe8o\x9a\x9b\x17u\xbe\xdc\xee'
Various alternatives are available, offering different return types (e.g.,
signed integers, tuples of unsigned integers) and optimized for different
architectures. For a comprehensive list of functions, refer to the
API Reference.
hashlib
-style hashers
mmh3
implements hasher objects with interfaces similar to those in hashlib
from the standard library, although they are still experimental. See
Hasher Classes
in the API Reference for more information.
Changelog
See Changelog
(latest version) for the complete changelog.
5.1.0 - 2025-01-25
Added
Removed
- Drop support for Python 3.8, as it has reached the end of life on 2024-10-07
(#117).
5.0.1 - 2024-09-22
Fixed
- Fix the issue that the package cannot be built from the source distribution
(#90).
5.0.0 - 2024-09-18
Added
- Add support for Python 3.13.
- Improve the performance of the
hash()
function with
METH_FASTCALL,
reducing the overhead of function calls. For data sizes between 1–2 KB
(e.g., 48x48 favicons), performance is 10%–20% faster. For smaller data
(~500 bytes, like 16x16 favicons), performance increases by approximately 30%
(#87). - Add
digest
functions that support the new buffer protocol
(PEP 688) as input
(#75).
These functions are implemented with METH_FASTCALL
too, offering improved
performance (#84). - Slightly improve the performance of the
hash_bytes()
function
(#88) - Add Read the Docs documentation
(#54).
- Document benchmark results
(#53).
Changed
- Backward-incompatible: The
seed
argument is now strictly validated to
ensure it falls within the range [0, 0xFFFFFFFF]. A ValueError
is raised
if the seed is out of range (#84). - Backward-incompatible: Change the constructors of hasher classes to
accept a buffer as the first argument
(#83).
- The type of flag argumens has been changed from
bool
to Any
(#84). - Change the format of CHANGELOG.md to conform to the
Keep a Changelog standard
(#63).
Deprecated
- Deprecate the
hash_from_buffer()
function.
Use mmh3_32_sintdigest()
or mmh3_32_uintdigest()
as alternatives
(#84).
Fixed
- Fix a reference leak in the
hash_from_buffer()
function
(#75). - Fix type hints (#76,
#77,
#84).
License
MIT, unless otherwise
noted within a file.
Frequently Asked Questions
Different results from other MurmurHash3-based libraries
By default, mmh3
returns signed values for the 32-bit and 64-bit versions
and unsigned values for hash128
due to historical reasons. To get the
desired result, use the signed
keyword argument.
Starting from version 4.0.0, mmh3
is endian-neutral, meaning that its
hash functions return the same values on big-endian platforms as they do on
little-endian ones. In contrast, the original C++ library by Appleby is
endian-sensitive. If you need results that comply with the original library on
big-endian systems, please use version 3.*.
For compatibility with Google Guava (Java),
see
https://stackoverflow.com/questions/29932956/murmur3-hash-different-result-between-python-and-java-implementation.
For compatibility with
murmur3 (Go), see
https://github.com/hajimes/mmh3/issues/46.
Handling errors with negative seeds
From the version 5.0.0, mmh3
functions accept only unsigned 32-bit integer
seeds to enable faster type-checking and conversion. However, this change may
cause issues if you need to calculate hash values using negative seeds within
the range of signed 32-bit integers. For instance,
Telegram-iOS uses
-137723950
as a hard-coded seed (bitwise equivalent to 4157243346
). To
handle such cases, you can convert a signed 32-bit integer to its unsigned
equivalent by applying a bitwise AND operation with 0xffffffff
. Here's an
example:
>>> mmh3.hash(b"quux", 4294967295)
258499980
>>> d = -1
>>> mmh3.hash(b"quux", d & 0xffffffff)
258499980
Alternatively, if the seed is hard-coded (as in the Telegram-iOS case), you can
precompute the unsigned value for simplicity.
Contributing Guidelines
See Contributing.
Authors
MurmurHash3 was originally developed by Austin Appleby and distributed under
public domain
https://github.com/aappleby/smhasher.
Ported and modified for Python by Hajime Senuma.
External Tutorials
High-performance computing
The following textbooks and tutorials are great resources for learning how to
use mmh3
(and other hash algorithms in general) for high-performance computing.
Internet of things
Shodan, the world's first
IoT search engine, uses
MurmurHash3 hash values for favicons
(icons associated with web pages). ZoomEye follows
Shodan's convention.
Calculating these values with mmh3
is useful for OSINT and cybersecurity activities.
How to Cite This Library
If you use this library in your research, it would be much appreciated it if
you would cite the following paper published in the
Journal of Open Source Software:
Hajime Senuma. 2025.
mmh3: A Python extension for MurmurHash3.
Journal of Open Source Software, 10(105):6124.
In BibTeX format:
@article{senumaMmh3PythonExtension2025,
title = {{mmh3}: A {Python} extension for {MurmurHash3}},
author = {Senuma, Hajime},
year = {2025},
month = jan,
journal = {Journal of Open Source Software},
volume = {10},
number = {105},
pages = {6124},
issn = {2475-9066},
doi = {10.21105/joss.06124},
copyright = {http://creativecommons.org/licenses/by/4.0/}
}
Related Libraries