Introducing Socket Firewall: Free, Proactive Protection for Your Software Supply Chain.Learn More
Socket
Book a DemoInstallSign in
Socket

airflow-provider-hightouch

Package Overview
Dependencies
Maintainers
1
Versions
16
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

airflow-provider-hightouch

Hightouch Provider for Airflow

pipPyPI
Version
4.0.0
Maintainers
1

Apache Airflow Provider for Hightouch

Provides an Airflow Operator and Hook for Hightouch. This allows the user to initiate a run for a sync from Airflow.

Installation

Pre-requisites: An environment running apache-airflow >= 1.10, including >= 2.

pip install airflow-provider-hightouch

Configuration

In the Airflow Connections UI, create a new connection for Hightouch.

  • Conn ID: hightouch_default
  • Conn Type: HTTP
  • Host: https://api.hightouch.com
  • Password: enter the API key for your workspace. You can generate an API key from your Workspace Settings

The Operator uses the hightouch_default connection id by default, but if needed, you can create additional Airflow Connections and reference them in the operator

Modules

HightouchTriggerSyncOperator

Starts a Hightouch Sync Run. Requires the sync_id or the sync_slug for the sync you wish to run.

Returns the sync_run_id of the sync it triggers.

The run is synchronous by default, and the task will be marked complete once the sync is successfully completed.

However, you can request a asynchronous request instead by passing synchronous=False to the operator.

If the API key is not authorized or if the request is invalid the task will fail. If a run is already in progress, a new run will be triggered following the completion of the existing run.

HightouchSyncRunSensor

Monitors a Hightouch Sync Run. Requires the sync_id and the sync_run_id of the sync you wish to monitor. To obtain the sync_run_id of a sync triggered in Airflow, we recommend using XComs to pass the return value of HightouchTriggerSyncOperator.

Examples

Creating a run is as simple as importing the operator and providing it with a sync_id. An example dag is available as well.

from airflow_provider_hightouch.operators.hightouch import HightouchTriggerSyncOperator

with DAG(....) as dag:
...
    my_task = HightouchTriggerSyncOperator(task_id="run_my_sync", sync_id="123")
    my_other_task = HightouchTriggerSyncOperator(task_id="run_my_sync", sync_slug="my-sync-slug")

Issues

Please submit issues and pull requests in our official repo: https://github.com/hightouchio/airflow-provider-hightouch

We are happy to hear from you, for any feedback please email the authors at pedram@hightouch.io.

Acknowledgements

Special thanks to Fivetran for their provider and Marcos Marx's Airbyte contribution in the core Airflow repo for doing this before we had to so we could generously learn from their hard work.

FAQs

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts