airflint

airflint

Enforce Best Practices for all your Airflow DAGs. ⭐

⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏

🧑‍🏫 Rules

• Use function-level imports instead of top-level imports¹² (see Top level Python Code)
• Use jinja template syntax instead of Variable.get (see Airflow Variables)

based on official Best Practices

Requirements

airflint is tested with:

	Main version (dev)	Released version (0.3.1-alpha)
Python	3.9, 3.10, 3.11.0-alpha - 3.11.0	3.9, 3.10
Apache Airflow	>= 2.0.0	>= 2.3.0

🚀 Get started

To install it from PyPI run:

pip install airflint

NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way UseJinjaVariableGet rule can detect all template_fields and airflint works as expected.

Then just call it like this:

pre-commit

Alternatively you can add the following repo to your pre-commit-config.yaml:

  - repo: https://github.com/feluelle/airflint
    rev: v0.3.1-alpha
    hooks:
      - id: airflint
        args: ["-a"]  # Use -a to apply the suggestions
        additional_dependencies:  # Add all package dependencies you have in your dags, preferable with version spec
          - apache-airflow
          - apache-airflow-providers-cncf-kubernetes

To complete the UseFunctionlevelImports rule, please add the autoflake hook after the airflint hook, as below:

  - repo: https://github.com/pycqa/autoflake
    rev: v1.4
    hooks:
      - id: autoflake
        args: ["--remove-all-unused-imports", "--in-place"]

This will remove unused imports.

❤️ Contributing

I am looking for contributors who are interested in..

• testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
• optimizing the ast traversing for existing rules
• adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
• documenting about what is being supported in particular by each rule
• defining supported airflow versions i.e. some rules are bound to specific Airflow features and version

For questions, please don't hesitate to open a GitHub issue.

Footnotes

1. There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule. ↩

2. To remove top-level imports after running UseFunctionLevelImports rule, use a tool such as autoflake. ↩