Tool to compile a DAG of a data processing pipeline from formatted TOML files.
By adhering to a portable, flexible standard, data science packages written by independent authors can be easily merged together. This foundational layer can then support and tie together tools and standards within and between disciplines.
Dagpiler is the first in a planned series of tools that will build on one another to standardize and facilitate common operations in data processing.
from dagpiler import compile_dag
from dagpiler.printer import print_dag, json_to_dag
# Compile the DAG from the various packages.
dag = compile_dag('package_name')
# Write the DAG to JSON file.
dag.print_dag('dag.json')
# Re-create the DAG from a JSON file.
dag_from_json = json_to_dag('dag.json')
Documentation: https://researchos.github.io/dagpiler/
FAQs
Compile a DAG representation of a data processing pipeline from TOML files.
We found that dagpiler demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Malicious npm package postcss-optimizer delivers BeaverTail malware, targeting developer systems; similarities to past campaigns suggest a North Korean connection.
Security News
CISA's KEV data is now on GitHub, offering easier access, API integration, commit history tracking, and automated updates for security teams and researchers.
Security News
Opengrep forks Semgrep to preserve open source SAST in response to controversial licensing changes.