Product
Introducing License Enforcement in Socket
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
An easy to use decorator for persistent memoization: like `functools.lrucache`, but results can be saved in any format to any storage.
An easy to use decorator for persistent memoization: like functools.lrucache
, but results persist between runs and can be stored in any format to any storage.
from perscache import Cache
cache = Cache()
@cache
def get_data():
...
pip install perscache
There are also some optional dependencies you need to install to use some of the features:
yaml
- for YAML serializationpyarrow
- for Parquet serializationpandas
- for CSV serializationgcsfs
- for GCS storagefrom perscache import Cache
cache = Cache()
counter = 0
@cache
def get_data():
print("Fetching data...")
global counter
counter += 1
return "abc"
print(get_data()) # the function is called
# Fetching data...
# abc
print(get_data()) # the cache is used
# abc
print(counter) # the function was called only once
# 1
@cache
def get_data(key):
print("The function has been called...")
return key
print(get_data("abc")) # the function has been called
# The function has been called...
# abc
print(get_data("fgh")) # the function has been called again
# The function has been called...
# fgh
print(get_data("abc")) # using the cache
# abc
@cache
def get_data(key):
print("This function has been changed...")
return key
print(get_data("abc")) # the function has been called again
# This function has been changed...
# abc
NOTE:
perscache
hashes the function arguments, its code and the name of the class of the serializer, so that the cache is invalidated when any of these change. You can use theignore_args
parameter to ignore changes in certain arguments of the decorated function.However, if you change the code of the serializer, the cache is not invalidated. (This is because we cannot hash the code of the serializer when in was made with the factory function
make_serializer
. See Make your own serialization and storage back-ends for more details.)
import datetime as dt
@cache(ttl=dt.timedelta(days=1))
def get_data():
"""This function will be cached for 1 day
and called again after this period expires."""
...
By specifying the arguments that should be ignored, you can still use the cache even in the values of these arguments have changed.
@cache(ignore="ignore_this")
def get_data(key, ignore_this):
print("The function has been called...")
return key
print(get_data("abc", "ignore_1")) # the function has been called
# The function has been called...
# abc
# using the cache although the the second argument is different
print(get_data("abc", "ignore_2"))
# abc
# set up serialization format and storage backend
cache = Cache(
serializer=JSONSerializer(),
storage=GoogleCloudStorage("/bucket/folder")
)
...
# change the default serialization format
@cache(serialization=PickleSerializer())
def get_data(key):
...
import os
from perscache import Cache, NoCache
from perscache.storage import LocalFileStorage
if os.environ.get["DEBUG"]:
cache = NoCache() # turn off caching in debug mode
else:
cache = (
GoogleCloudStorage("/bucket/folder")
if os.environ.get["GOOGLE_PROJECT_NAME"] # if running in the cloud
else LocalFileStorage()
)
@cache
def function():
...
Use human-readable serialization (JSONSerializer
, YAMLSerializer
, CSVSerializer
) and a file storage (LocalFileStorage
, GoogleCloudStorage
) to inspect cached results.
When using LocalFileStorage(max_size=...)
or GoogleCloudStorage(max_size=...)
, the least recently used cache entries are automatically removed to keep the total cache size with the max_size
limit.
Although you can use the standard CloudPickleSerializer()
for almost any type of data, sometimes you want to inspect the results of a decorated function by lookin into the cache files. This requires the data to be serialized in a human-readable format. But the included human-readable serializers (JSONSerializer()
, YAMLSerializer()
, CSVSerializer()
) sometimes cannot process complex objects.
To see which serializers are compatible with which data types, see the compatibility.py file.
That's when making your own serializer comes in handy.
To do this, you should:
Serializer
class and override the abstract methods. You should also provide the extension
class variable that specifies the file extension.Cache
class.from perscache.serializers import Serializer
class MySerializer(Serializer):
extension = "data"
def dumps(self, data: Any) -> bytes:
...
def loads(self, data: bytes) -> Any:
...
cache = Cache(serializer=MySerializer())
You can also use the perscache.serializers.make_serializer()
function to create a serializer for a given data type.
import pyrogram
from perscache.serializers import make_serializer
PyrogramSerializer = make_serializer(
"PyrogramSerializer",
"pyro",
dumps_fn = lambda data: str(data).encode("utf-8"),
loads_fn = lambda data: eval(data.decode("utf-8")),
)
cache = Cache(serializer=PyrogramSerializer())
@cache
async def some_pyrogram_func() -> pyrogram.Message:
...
Making a custom storage backed is similar:
class MyStorage(Storage):
def read(self, path, deadline: datetime.datetime) -> bytes:
"""Read the file at the given path and return its contents as bytes.
If the file does not exist, raise FileNotFoundError. If the file is
older than the given deadline, raise CacheExpired.
"""
...
def write(self, path, data: bytes) -> None:
"""Write the file at the given path."""
...
cache = Cache(storage=MyStorage())
You can also derive your storage class from perscache.storage.FileStorage
if you are building a filesystem-based storage back-end. Refer to the storage.py file for more information.
FAQs
An easy to use decorator for persistent memoization: like `functools.lrucache`, but results can be saved in any format to any storage.
We found that perscache demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Ensure open-source compliance with Socket’s License Enforcement Beta. Set up your License Policy and secure your software!
Product
We're launching a new set of license analysis and compliance features for analyzing, managing, and complying with licenses across a range of supported languages and ecosystems.
Product
We're excited to introduce Socket Optimize, a powerful CLI command to secure open source dependencies with tested, optimized package overrides.