distify
Wrapper around Ray for easy distributed processing in Python.
Disclaimer: I use it for some of my projects, but there is no guarantee that the code will work as expected or that future versions will be backward compatible.
Install
pip install distify --upgrade
Features
- Multiple backends: Ray, Multiprocessing, Multithreading, Sequential.
- Logging.
- Progress bar.
- Can run in local or in multiple nodes.
- Individual timeout for map applications.
- Resume from checkpointing.
- Hydra integration.
Quickstart/usage
Please refer to this example: https://github.com/jordiae/distify/tree/main/examples/basic
Documentation
TODO
LICENSE
MIT
Author
Jordi Armengol Estapé