Pain-Free redis pipelining in python.
|BuildStatus| |CoverageStatus| |Version| |Python| |Docs|
RedPipe is a wrapper around the pipeline component of redis-py <https://redis-py.readthedocs.io>_.
It makes it easy to reduce network round trips when talking to Redis.
For more general information about redis pipelining, see the official redis documentation <https://redis.io/topics/pipelining>_.
Use RedPipe to build pipelined redis calls in a modular reusable way. Rewrite your existing application calls via redis-py into efficient batches with only minimal changes.
The redpipe module requires Python 2.7 or higher.
To install, use pip:
.. code-block::
$ pip install redpipe
or from source:
.. code-block::
$ python setup.py install
To use redpipe, You need to bind your redis client instance to RedPipe. Use the standard redis-py client.
.. code-block:: python
client = redis.Redis()
redpipe.connect_redis(client)
You only need to do this setup once during application bootstrapping.
Using RedPipe is easy. We can pipeline multiple calls to redis and assign the results to variables. This makes pipeline code look and feel similar to the regular interface of redis-py.
.. code-block:: python
with redpipe.pipeline() as pipe:
foo = pipe.incr('foo')
bar = pipe.incr('bar)
pipe.execute()
print([foo, bar])
Find RedPipe documentation on Read the Docs <http://redpipe.readthedocs.io/en/latest/>_.
.. |BuildStatus| image:: https://travis-ci.org/72squared/redpipe.svg?branch=master :target: https://app.travis-ci.com/github/72squared/redpipe
.. |CoverageStatus| image:: https://coveralls.io/repos/github/72squared/redpipe/badge.svg?branch=master :target: https://coveralls.io/github/72squared/redpipe?branch=master
.. |Version| image:: https://badge.fury.io/py/redpipe.svg :target: https://badge.fury.io/py/redpipe
.. |Python| image:: https://img.shields.io/badge/python-2.7,3.4,pypy-blue.svg :target: https://pypi.python.org/pypi/redpipe/
.. |Docs| image:: https://readthedocs.org/projects/redpipe/badge/?version=latest :target: http://redpipe.readthedocs.org/en/latest/ :alt: Documentation Status
Modify Protocol import to allow python 3.7 to work.
Add support for type hints. Remove any python 2 references. Remove dependency on "six" library now that only python3 is supported.
Update the docs to reflect deprecation of redis-py-cluster. Use redis-py instead.
Maintain compatibility with latest version of redis-py, now that it natively supports redis cluster.
Bugfix. enable_threads and disable_threads behavior were swapped.
Added a test to confirm correct behavior.
Fixing an issue where iterators getting passed in to multi key operations were being treated as a single key.
StringListField should return an empty string for an empty list encoding. Sending None to the redis results in bad stuff.
Relaxing validation for fields a little bit and try to just do the right thing. Only blow up if we really can't coerce to what we think is a correct value.
Make IntegerField a little more forgiving and allow float values to be passed in. Auto-convert to integers. Makes lots of things easier.
Introduce Keyspace.super_pipe property. Allows multiple pipelined functions to share the same parent pipeline. Don't use this unless you know what you're doing. We will work out the documentation as we explore the need for this further and standardize the api.
Bug fix: pickling future objects with a result that evaluates to false causes unpickling to not call setstate. This causes the future to raise an exception on unpickling when trying to use the future result because no state has been set.
In python documentation it states: "Note For new-style classes, if getstate() returns a false value, the setstate() method will not be called."
reference: https://docs.python.org/3/library/pickle.html
The fix is to return a dictionary with the result in getstate and expect one in setstate so the value never evaluates to false.
z[rev]range[byscore] functions should return list of tuples instead of list of lists. fix provided by Max Greenblatt, thanks! This makes the interface more closely match the behavior in redis-py which acts as a reference spec for redpipe commands. This is a minor change and probably won't break anything on the client, but incrementing the minor version, just in case.
enable key by name as attribute.
If you set field_attr_on to true on a Struct, you can
access fields as attributes of the struct. Not recommended,
but there may be certain cases where it can be desirable.
Use with caution.
Support for Strings stored in a sharded Hash.
Adding mget on String keyspace data.
adding required fields and defaults on struct.
rename _memberparse to memberparse on Hash class to be consistent
with naming conventions of keyparse and valueparse variables.
Also allows access of this variable outside the class which is fine.
ttl on struct objects. documentation fixes.
to avoid confusion, make the eval command in keyspaces match the interface defined in redis-py.
adding support for zcount (somehow missed it before).
fixing a bug in named connections in structs and keyspaces. fixes issue #2
enable pfcount when using rediscluster.
adding an nx option to struct to allow changes to be set only if the properties don't already exist.
No substantive changes from 1.0.0rc3. Updating notes and removing beta flags.
Use threads by default when talking to multiple backends in the same pipeline.
You can disable this behavior with redpipe.disable_threads().
Minor changes.
This marks the first RC. There are a few breaking changes, mostly easily fixed.
More breaking changes to Struct. Solidifying the api. Making important simplifications. This will make it easier to explain and document.
After experimenting with some things, simplifying Struct back down. Some of the methods in Struct will break. Easier to explain with fewer methods and can still do everything I need to.
This release also improves the documentation on Struct. I hadn't bothered much up until this point. The interface was still solidifying. Starting to get to a stable place there.
Breaking changes in this release as well.
Can only access data from a struct object like you would a dictionary.
This is an important step because it disambiguates commands from data.
And it enforces one consistent way to access data.
All the methods on the Struct give it a dictionary interface.
Easier to explain the mental model this way.
redpipe.Struct.BIG REFACTOR. key no longer part of the constructor of Keyspace objects. Instead, you pass the key name to the method. This keeps the api identical in arguments in redis-py. It also allows me to support multi-key operations. This is a breaking change.
Releases prior to 1.0.0 are considered beta. The api is not officially supported. We make no guarantees about backward compatibility.
Releases less than 0.1.0 in this project are considered early alpha and don't deserve special mention.
FAQs
Easy Redis pipelines
We found that redpipe demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.