pylightnix

Pylightnix

Pylightnix is a minimalistic Python library for managing filesystem-based immutable data objects, inspired by Purely Functional Software Deployment Model thesis by Eelco Dolstra and the Nix package manager.

The library may be thought as of low-level API for creating caching wrappers for a subset of Python functions. In particular, Pylightnix allows user to

• Prepare (or, in our terms - instantiate) the computation plan aimed at creating a tree of linked immutable stage objects, stored in the filesystem.
• Implement (in our terms - realize) the prepared computation plan, access the resulting artifacts. Pylightnix is able to handle non-deterministic results of the computation. As one example, it is possible to define a stage depending on best top-10 instances (in a user-defined sense) of prerequisite stages.
• Handle changes in the computation plan, re-use the existing artifacts whenever possible.
• Gain full control over all aspects of the cached data including the garbage-collection.

Documentation

QuickStart [PDF] | API Referece [MD]

Demos:

• GNU Hello, turns Pylightnix into a toy package manager to build the GNU Hello from sources.
• MDRUN, evaluate code sections in a Markdown document, and cache the results betwee runs.

(Outdated)

Manual [PDF] |

• Ultimatum tutorial, managing experiments.
• MNIST demo shows some machine learning specific.
• REPL demo illustrates how to debug stage sequences.