Documentation | PyPI | GitHub | Paper
Jobflow is a free, open-source library for writing and executing workflows. Complex workflows can be defined using simple python functions and executed locally or on arbitrary computing resources using the jobflow-remote or FireWorks workflow managers.
Some features that distinguish jobflow are dynamic workflows, easy compositing and connecting of workflows, and the ability to store workflow outputs across multiple databases.
jobflow is intended to be a friendly workflow software that is easy to get started with, but flexible enough to handle complicated use cases.
Some of its features include:
Workflows in jobflows are made up of two main components:
Job is an atomic computing job. Essentially any python function can be Job,
provided its input and return values can be serialized to json. Anything returned by the job is
considered an "output" and is stored in the jobflow database.Flow is a collection of Job objects or other Flow objects. The connectivity
between jobs is determined automatically from the job inputs. The execution order
of jobs is automatically determined based on their connectivity.Python functions can be easily converted in to Job objects using the @job decorator.
In the example below, we define a job to add two numbers.
from jobflow import job, Flow
@job
def add(a, b):
return a + b
add_first = add(1, 5)
add_second = add(add_first.output, 5)
flow = Flow([add_first, add_second])
flow.draw_graph().show()
The output of the job is accessed using the output attribute. As the job has not
yet been run, output contains be a reference to a future output. Outputs can be used
as inputs to other jobs and will be automatically "resolved" before the job is
executed.
Finally, we created a flow using the two Job objects. The connectivity between
the jobs is determined automatically and can be visualised using the flow graph.
jobflow is a Python 3.9+ library and can be installed using pip.
pip install jobflow
To get a first glimpse of jobflow, we suggest that you follow our quickstart tutorial. Later tutorials delve into the advanced features of jobflow.
Ask questions about jobflow on the jobflow support forum. If you've found an issue with jobflow, please submit a bug report on GitHub Issues.
Track changes to jobflow through the changelog.
We greatly appreciate any contributions in the form of a pull request. Additional information on contributing to jobflow can be found here. We maintain a list of all contributors here.
jobflow is released under a modified BSD license; the full text can be found here.
If you use Jobflow in your work, please cite it as follows:
Jobflow was designed by Alex Ganose, Anubhav Jain, Gian-Marco Rignanese, David Waroquiers, and Guido Petretto. Alex Ganose implemented the first version of the package. Later versions have benefited from the contributions of several research groups. A full list of contributors is available here.
FAQs
jobflow is a library for writing computational workflows
We found that jobflow demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
A look at the top trends in how threat actors are weaponizing open source packages to deliver malware and persist across the software supply chain.
Security News
ESLint now supports HTML linting with 48 new rules, expanding its language plugin system to cover more of the modern web development stack.
Security News
CISA is discontinuing official RSS support for KEV and cybersecurity alerts, shifting updates to email and social media, disrupting automation workflows.