python-redis-lock
Advanced tools
.. list-table:: :stub-columns: 1
* - docs
- |docs|
* - tests
- |github-actions| |coveralls| |codecov|
* - package
- |version| |wheel| |supported-versions| |supported-implementations| |commits-since|
.. |docs| image:: https://readthedocs.org/projects/python-redis-lock/badge/?style=flat :target: https://readthedocs.org/projects/python-redis-lock/ :alt: Documentation Status .. |github-actions| image:: https://github.com/ionelmc/python-redis-lock/actions/workflows/github-actions.yml/badge.svg :alt: GitHub Actions Build Status :target: https://github.com/ionelmc/python-redis-lock/actions .. |coveralls| image:: https://coveralls.io/repos/github/ionelmc/python-redis-lock/badge.svg?branch=master :alt: Coverage Status :target: https://coveralls.io/github/ionelmc/python-redis-lock?branch=master .. |codecov| image:: https://codecov.io/gh/ionelmc/python-redis-lock/branch/master/graphs/badge.svg?branch=master :alt: Coverage Status :target: https://app.codecov.io/github/ionelmc/python-redis-lock .. |version| image:: https://img.shields.io/pypi/v/python-redis-lock.svg :alt: PyPI Package latest release :target: https://pypi.org/project/python-redis-lock .. |wheel| image:: https://img.shields.io/pypi/wheel/python-redis-lock.svg :alt: PyPI Wheel :target: https://pypi.org/project/python-redis-lock .. |supported-versions| image:: https://img.shields.io/pypi/pyversions/python-redis-lock.svg :alt: Supported versions :target: https://pypi.org/project/python-redis-lock .. |supported-implementations| image:: https://img.shields.io/pypi/implementation/python-redis-lock.svg :alt: Supported implementations :target: https://pypi.org/project/python-redis-lock .. |commits-since| image:: https://img.shields.io/github/commits-since/ionelmc/python-redis-lock/v4.0.1.svg :alt: Commits since latest release :target: https://github.com/ionelmc/python-redis-lock/compare/v4.0.1...master
Lock context manager implemented via redis SETNX/BLPOP.
Interface targeted to be exactly like threading.Lock <https://docs.python.org/2/library/threading.html#threading.Lock>_.
Because we don't want to require users to share the lock instance across processes you will have to give them names.
.. code-block:: python
from redis import Redis
conn = Redis()
import redis_lock
lock = redis_lock.Lock(conn, "name-of-the-lock")
if lock.acquire(blocking=False):
print("Got the lock.")
lock.release()
else:
print("Someone else has the lock.")
.. code-block:: python
conn = StrictRedis()
with redis_lock.Lock(conn, "name-of-the-lock"):
print("Got the lock. Doing some work ...")
time.sleep(5)
You can also associate an identifier along with the lock so that it can be retrieved later by the same process, or by a different one. This is useful in cases where the application needs to identify the lock owner (find out who currently owns the lock).
.. code-block:: python
import socket
host_id = "owned-by-%s" % socket.gethostname()
lock = redis_lock.Lock(conn, "name-of-the-lock", id=host_id)
if lock.acquire(blocking=False):
assert lock.locked() is True
print("Got the lock.")
lock.release()
else:
if lock.get_owner_id() == host_id:
print("I already acquired this in another process.")
else:
print("The lock is held on another machine.")
The dogpile is also known as the thundering herd effect or cache stampede. Here's a pattern to avoid the problem without serving stale data. The work will be performed a single time and every client will wait for the fresh data.
To use this you will need django-redis <https://github.com/jazzband/django-redis>_, however, python-redis-lock
provides you a cache backend that has a cache method for your convenience. Just install python-redis-lock like
this:
.. code-block:: bash
pip install "python-redis-lock[django]"
Now put something like this in your settings:
.. code-block:: python
CACHES = {
'default': {
'BACKEND': 'redis_lock.django_cache.RedisCache',
'LOCATION': 'redis://127.0.0.1:6379/1',
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient'
}
}
}
.. note::
If using a django-redis < 3.8.x, you'll probably need redis_cache
which has been deprecated in favor to django_redis. The redis_cache
module is removed in django-redis versions > 3.9.x. See django-redis notes <https://github.com/jazzband/django-redis#configure-as-cache-backend>_.
This backend just adds a convenient .lock(name, expire=None) function to django-redis's cache backend.
You would write your functions like this:
.. code-block:: python
from django.core.cache import cache
def function():
val = cache.get(key)
if not val:
with cache.lock(key):
val = cache.get(key)
if not val:
# DO EXPENSIVE WORK
val = ...
cache.set(key, value)
return val
In some cases, the lock remains in redis forever (like a server blackout / redis or application crash / an unhandled
exception). In such cases, the lock is not removed by restarting the application. One solution is to turn on the
auto_renewal parameter in combination with expire to set a time-out on the lock, but let Lock() automatically
keep resetting the expire time while your application code is executing:
.. code-block:: python
# Get a lock with a 60-second lifetime but keep renewing it automatically
# to ensure the lock is held for as long as the Python process is running.
with redis_lock.Lock(conn, name='my-lock', expire=60, auto_renewal=True):
# Do work....
Another solution is to use the reset_all() function when the application starts:
.. code-block:: python
# On application start/restart
import redis_lock
redis_lock.reset_all()
Alternatively, you can reset individual locks via the reset method.
Use these carefully, if you understand what you do.
redis_lock will use 2 keys for each lock named <name>:
lock:<name> - a string value for the actual locklock-signal:<name> - a list value for signaling the waiters when the lock is releasedThis is how it works:
.. image:: https://raw.githubusercontent.com/ionelmc/python-redis-lock/master/docs/redis-lock%20diagram%20(v3.0).png :alt: python-redis-lock flow diagram
https://python-redis-lock.readthedocs.io/en/latest/
To run the all tests run::
.png){kind=link}
tox
:OS: Any :Runtime: Python 2.7, 3.3 or later, or PyPy :Services: Redis 2.6.12 or later.
bbangert/retools <https://github.com/bbangert/retools/blob/0.4/retools/lock.py>_ - acquire does spinloopdistributing-locking-python-and-redis <https://chris-lamb.co.uk/posts/distributing-locking-python-and-redis>_ - acquire does pollingcezarsa/redis_lock <https://github.com/cezarsa/redis_lock/blob/0.2.0/redis_lock/__init__.py>_ - acquire does not blockandymccurdy/redis-py <https://github.com/andymccurdy/redis-py/blob/3.5.3/redis/lock.py>_ - acquire does spinloopmpessas/python-redis-lock <https://github.com/mpessas/python-redis-lock/blob/b512eef0fc5e1e2e82a6a31f65cd88c2c37dfe4b/redislock/lock.py>_ - blocks fine but no expirationbrainix/pottery <https://github.com/brainix/pottery/blob/v1.1.5/pottery/redlock.py>_ - acquire does spinloop#94 <https://github.com/ionelmc/python-redis-lock/pull/94>_.#110 <https://github.com/ionelmc/python-redis-lock/pull/110>_.blocking argument to Lock.__init__ to allow this behavior when using it as
a context manager.
Contributed by Andrew Petriv in #102 <https://github.com/ionelmc/python-redis-lock/pull/102>_.redis_lock.refresh.thread.* loggers with a single redis_lock.refresh.thread logger.Made logger names more specific. Now can have granular filtering on these new logger names:
redis_lock.acquire (emits DEBUG messages)redis_lock.acquire (emits WARN messages)redis_lock.acquire (emits INFO messages)redis_lock.refresh.thread.start (emits DEBUG messages)redis_lock.refresh.thread.exit (emits DEBUG messages)redis_lock.refresh.start (emits DEBUG messages)redis_lock.refresh.shutdown (emits DEBUG messages)redis_lock.refresh.exit (emits DEBUG messages)redis_lock.release (emits DEBUG messages)Contributed by Salomon Smeke Cohen in #80 <https://github.com/ionelmc/python-redis-lock/pull/80>_..
Fixed few CI issues regarding doc checks.
Contributed by Salomon Smeke Cohen in #81 <https://github.com/ionelmc/python-redis-lock/pull/81>_..
Improved timeout/expire validation so that:
timeout and expire are converted to Noneif they are falsy. Previously onlyNone`` disabled these options, other falsy
values created buggy situations.timeout greater than expire is now allowed, if auto_renewal is set to True. Previously a TimeoutTooLarge error
was raised.
See #74 <https://github.com/ionelmc/python-redis-lock/issues/74>_..timeout or expire are disallowed. Previously such values were allowed, and created buggy situations.
See #73 <https://github.com/ionelmc/python-redis-lock/issues/73>_..Updated benchmark and examples.
Removed the custom script caching code. Now the register_script method from the redis client is used.
This will fix possible issue with redis clusters in theory, as the redis client has some specific handling for that.
locked method. Contributed by Artem Slobodkin in #72 <https://github.com/ionelmc/python-redis-lock/pull/72>_..#71 <https://github.com/ionelmc/python-redis-lock/issues/71>_..#64 <https://github.com/ionelmc/python-redis-lock/issues/64>_..warnings API. Contributed by Julie MacDonell in
#54 <https://github.com/ionelmc/python-redis-lock/pull/54>_..auto_renewal option in RedisCache.lock (the Django cache backend wrapper). Contributed by c
in #55 <https://github.com/ionelmc/python-redis-lock/pull/55>_..decode_responses=True. Lock keys are pure ascii now.#38 <https://github.com/ionelmc/python-redis-lock/pull/38>_..id to acquire. Previously it assumed that if you specify the id then the lock was already
acquired. See #44 <https://github.com/ionelmc/python-redis-lock/issues/44>. and
#39 <https://github.com/ionelmc/python-redis-lock/issues/39>..strict=False. Normally you're expected to pass in an instance
of redis.StrictRedis.locked_get_or_set to Django cache backend.#33 <https://github.com/ionelmc/python-redis-lock/pull/33>_..release so that it expires signal-keys immediately. Contributed by Andrew Pashkin in #28 <https://github.com/ionelmc/python-redis-lock/pull/28>_..reset or reset_all) will release the lock. If there's someone waiting on the reset lock now it will
acquire it. Contributed by Andrew Pashkin in #29 <https://github.com/ionelmc/python-redis-lock/pull/29>_..extend method on Lock objects. Contributed by Andrew Pashkin in #24 <https://github.com/ionelmc/python-redis-lock/pull/24>_..release method. Contributed by Andrew Pashkin in #22 <https://github.com/ionelmc/python-redis-lock/pull/22>_..acquire(block=True) handling when expire option was used (it wasn't blocking indefinitely). Contributed by
Tero Vuotila in #35 <https://github.com/ionelmc/python-redis-lock/pull/35>_..release to check if lock was acquired with he same id. If not, NotAcquired will be raised.
Previously there was just a check if it was acquired with the same instance (self._held).
BACKWARDS INCOMPATIBLEforce option from release - it wasn't really necessary and it only encourages sloppy programming. See
#25 <https://github.com/ionelmc/python-redis-lock/issues/25>_..
BACKWARDS INCOMPATIBLEtimeout option. Contributed by Victor Torres in #20 <https://github.com/ionelmc/python-redis-lock/pull/20>_..auto_renewal option. Contributed by Nick Groenen in #18 <https://github.com/ionelmc/python-redis-lock/pull/18>_..AlreadyAcquired and NotAcquired.Lock.token to Lock.id. Now only allowed to be set via constructor. Contributed by Jardel Weyrich in #11 <https://github.com/ionelmc/python-redis-lock/pull/11>_..reset_all functionality. Contributed by Yokotoka in #7 <https://github.com/ionelmc/python-redis-lock/pull/7>_..Lock.reset functionality.Lock.token attribute.???FAQs
Lock context manager implemented via redis SETNX/BLPOP.
We found that python-redis-lock demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
GitHub account BufferZoneCorp published sleeper packages that later added credential theft, GitHub Actions tampering, fake go wrappers, and SSH persistence.
Research
/Security News
Socket found a malicious Intercom PHP package on Packagist using Composer plugin execution to steal credentials and spread across ecosystems.
Research
/Security News
Compromised intercom-client@7.0.4 npm package is tied to the ongoing Mini Shai-Hulud worm attack targeting developer and CI/CD secrets.