dbt-cloud-cli is a command line interface for dbt Cloud API. It abstracts the REST API calls in an easy-to-use interface that can be incorporated into automated and manual (ad-hoc) workloads. Here are some example use cases for dbt-cloud-cli:
catalog.json) from dbt Cloud.dbt-cloud-cli has been tested with the following Python versions:
Installation from PyPI:
pip install dbt-cloud-cli
Running in Docker:
docker run datamie/dbt-cloud-cli:latest
The following environment variables are used as argument defaults:
DBT_CLOUD_HOST (--dbt-cloud-host): dbt Cloud host (cloud.getdbt.com (multi-tenant instance) by default if the environment variable is not set)DBT_CLOUD_API_TOKEN (--api-token): API authentication keyDBT_CLOUD_ACCOUNT_ID (--account-id): Numeric ID of the dbt Cloud accountDBT_CLOUD_JOB_ID (--job-id): Numeric ID of a dbt Cloud jobFor more information on a command, run dbt-cloud <command> --help. For more information on the API endpoints, see dbt Cloud API V3 docs and dbt Cloud Metadata API docs.
| Group | Command | Implemented | API endpoint |
|---|---|---|---|
| Account | dbt-cloud account get | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/ |
| Account | dbt-cloud account list | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/ |
| Audit log | dbt-cloud audit-log get | ✅ | GET https://{dbt_cloud_host}/api/v3/audit-logs/ |
| Project | dbt-cloud project create | ✅ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/ |
| Project | dbt-cloud project delete | ✅ | DELETE https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{id}/ |
| Project | dbt-cloud project get | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{id}/ |
| Project | dbt-cloud project list | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/ |
| Project | dbt-cloud project update | ✅ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{id}/ |
| Environment | dbt-cloud environment create | ✅ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/environments/ |
| Environment | dbt-cloud environment delete | ✅ | DELETE https://{dbt_cloud_host}/api/v3/accounts/{account_id}/environments/{id}/ |
| Environment | dbt-cloud environment get | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/environments/{id}/ |
| Environment | dbt-cloud environment list | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/environments/ |
| Environment | dbt-cloud environment update | ❌ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/environments/{id}/ |
| Connection | dbt-cloud connection create | ✅ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{project_id}/connections/ |
| Connection | dbt-cloud connection delete | ✅ | DELETE https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{project_id}/connections/{id}/ |
| Connection | dbt-cloud connection get | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{project_id}/connections/{id}/ |
| Connection | dbt-cloud connection list | ✅ | GET https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{project_id}/connections/ |
| Connection | dbt-cloud connection update | ❌ | POST https://{dbt_cloud_host}/api/v3/accounts/{account_id}/projects/{project_id}/connections/{id}/ |
| Repository | dbt-cloud repository create | ❌ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/repositories/ |
| Repository | dbt-cloud repository delete | ❌ | DELETE https://{dbt_cloud_host}/api/v2/accounts/{account_id}/repositories/{id}/ |
| Repository | dbt-cloud repository get | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/repositories/{id}/ |
| Repository | dbt-cloud repository list | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/repositories/ |
| Job | dbt-cloud job create | ✅ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/ |
| Job | dbt-cloud job delete | ✅ | DELETE https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/{id}/ |
| Job | dbt-cloud job delete-all | ✅ | Uses a composition of one or more endpoints |
| Job | dbt-cloud job get | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/{id}/ |
| Job | dbt-cloud job list | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/ |
| Job | dbt-cloud job run | ✅ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/{job_id}/run/ |
| Job | dbt-cloud job update | ❌ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/{id}/ |
| Job | dbt-cloud job get-artifact | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/jobs/{job_id}/artifacts/{remainder} |
| Job | dbt-cloud job export | ✅ | Uses a composition of one or more endpoints |
| Job | dbt-cloud job import | ✅ | Uses a composition of one or more endpoints |
| Run | dbt-cloud run get | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/runs/{id}/ |
| Run | dbt-cloud run list | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/runs/ |
| Run | dbt-cloud run cancel | ✅ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/runs/{run_id}/cancel/ |
| Run | dbt-cloud run cancel-all | ✅ | Uses a composition of one or more endpoints |
| Run | dbt-cloud run list-artifacts | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/runs/{run_id}/artifacts/ |
| Run | dbt-cloud run get-artifact | ✅ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/runs/{run_id}/artifacts/{remainder} |
| Run | dbt-cloud run get-step | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/steps/{id}/ |
| User | dbt-cloud user get | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/users/{id}/ |
| User | dbt-cloud user list | ❌ | GET https://{dbt_cloud_host}/api/v2/accounts/{account_id}/users/ |
| User | dbt-cloud user update | ❌ | POST https://{dbt_cloud_host}/api/v2/accounts/{account_id}/users/{id}/ |
| Metadata | dbt-cloud metadata query | ✅ | POST https://{dbt_cloud_host}/graphql/ |
This command retrieves dbt Cloud account information.
dbt-cloud account get --account-id 123456
This command retrieves all available dbt Cloud accounts.
dbt-cloud account list
❗ Available for Enterprise accounts only.
This command retrieves audit logs for the dbt Cloud account.
dbt-cloud audit-log get --logged-at-start 2022-05-01 --logged-at-end 2022-05-07 --limit 1
This command creates a new dbt Cloud project in a given account.
dbt-cloud project create --name "My project"
This command deletes a dbt Cloud project in a given account.
dbt-cloud project delete --project-id 273731
This command retrieves dbt Cloud project information.
dbt-cloud project get --project-id 123457
This command returns a list of projects in the account.
dbt-cloud project list
This command updates a project in a given account.
dbt-cloud project update --project-id 273745 --name "My project renamed"
This command a new dbt Cloud environment in a given project.
dbt-cloud environment create --account-id 123456 --project-id 123457 --name "My environment" --dbt-version "1.5.0-latest"
This command deletes a dbt Cloud environment in a given project.
dbt-cloud environment delete --account-id 123456 --project-id 123457 --environment-id 40480
This command retrieves environments in a given project.
dbt-cloud environment list --account-id 123456 --project-id 123457 --limit 1
This command retrieves information about an environment in a given project.
dbt-cloud environment get --account-id 123456 --project-id 123457 --environment-id 67890
This command creates a new database connection in a given project. Supported connection types:
snowflake: Connection to a Snowflake database. Has inout validation for connection parameters.bigquery: Connection to a Google BigQuery database. No input validation.postgres: Connection to a PostgreSQL database. No input validation.redshift: Connection to an Amazon Redshift database. No input validation.adapter: Connection to a database using a custom dbt Cloud adapter. No input validation.dbt-cloud connection create --account-id 54321 --project-id 123467 --name Snowflake --type snowflake --account snowflake_account --database analytics --warehouse transforming --role transformer --allow-sso False --client-session-keep-alive False
This command deletes a database connection in a given project.
dbt-cloud connection delete --account-id 54321 --project-id 123467 --connection-id 56901
This command retrievies details of dbt Cloud database connections in a given project.
dbt-cloud connection list --account-id 54321 --project-id 123467 --limit 1
This command retrievies the details of a dbt Cloud database connection.
dbt-cloud connection get --account-id 54321 --project-id 123467 --connection-id 56901
This command triggers a dbt Cloud job run and returns a run status JSON response.
>> dbt-cloud job run --job-id 43167 --cause "My first run!" --steps-override '["dbt seed", "dbt run"]' --wait
Job 43167 run 34929305: QUEUED ...
Job 43167 run 34929305: QUEUED ...
Job 43167 run 34929305: QUEUED ...
Job 43167 run 34929305: STARTING ...
Job 43167 run 34929305: RUNNING ...
Job 43167 run 34929305: SUCCESS ...
This command returns the details of a dbt Cloud job.
dbt-cloud job get --job-id 43167
This command returns a list of jobs in the account.
dbt-cloud job list --account-id 123456 --project-id 123457 --limit 2
This command creates a job in a dbt Cloud project.
dbt-cloud job create --project-id 12345 --environment-id 49819 --name "Create job" --execute-steps '["dbt seed", "dbt run"]'
This command deletes a job in a dbt Cloud project.
dbt-cloud job delete --job-id 48474
💡 This is a composition of one or more base commands.
This command fetches all jobs on the account, deletes them one-by-one after user confirmation via prompt and prints out the job delete responses.
>> dbt-cloud job delete-all --keep-jobs "[43167, 49663]"
Jobs to delete: [54658, 54659]
Delete job 54658? [y/N]: yes
Job 54658 was deleted.
Delete job 54659? [y/N]: yes
Job 54659 was deleted.
💡 This is a composition of one or more base commands.
This command exports a dbt Cloud job as JSON to a file and can be used in conjunction with dbt-cloud job import to copy jobs between dbt Cloud projects.
dbt-cloud job export > job.json
💡 This is a composition of one or more base commands.
This command imports a dbt Cloud job from exported JSON. You can use JSON manipulation tools (e.g., jq) to modify the job definition before importing it.
dbt-cloud job export > job.json
cat job.json | jq '.environment_id = 49819 | .name = "Imported job"' | dbt-cloud job import
This command returns the details of a dbt Cloud run.
dbt-cloud run get --run-id 36053848
This command returns a list of runs in the account.
dbt-cloud run list --limit 2
This command cancels a dbt Cloud run. A run can be 'cancelled' irregardless of it's previous status. This means that you can send a request to cancel a previously successful / errored run (and nothing happens practically) and the response status would be similar to cancelling a currently queued or running run.
dbt-cloud run cancel --run-id 36053848
💡 This is a composition of one or more base commands.
This command fetches all runs on the account, cancels them one-by-one after user confirmation via prompt and prints out the run cancellation responses.
You should typically use this with a --status arg of either Running or Queued as cancellations can be requested against all runs. Without this, you will effectively be trying to cancel all runs that had ever been scheduled in the project irregardless of its' current status (which could take a long time if your project has had a lot of previous runs).
>> dbt-cloud run cancel-all --status Running
Runs to cancel: [36053848]
Cancel run 36053848? [y/N]: yes
Run 36053848 has been cancelled.
This command fetches a list of artifact files generated for a completed run.
dbt-cloud run list-artifacts --run-id 36053848
This command fetches an artifact file from a completed run. Once a run has completed, you can use this command to download the manifest.json, run_results.json or catalog.json files from dbt Cloud. These artifacts contain information about the models in your dbt project, timing information around their execution, and a status message indicating the result of the model build.
dbt-cloud run get-artifact --run-id 36053848 --path manifest.json > manifest.json
This command queries the dbt Cloud Metadata API using GraphQL.
dbt-cloud metadata query -f query.graphql
An alternative way of using the command without saving the GraphQL query to a file is to pipe it to dbt-cloud metadata query.
>> echo '{
model(jobId: 49663, uniqueId: "model.jaffle_shop.customers") {
parentsModels {
runId
uniqueId
executionTime
}
parentsSources {
runId
uniqueId
state
}
}
}' | dbt-cloud metadata query
The utilities listed here are for demonstration purposes only and are subject to change. In order to use the demo utilities you need to install the dbt-cloud-cli with extra demo dependencies:
pip install dbt-cloud-cli[demo]
An interactive CLI application for exploring catalog.json artifacts.
>> latest_run_id=$(dbt-cloud run list --job-id $DBT_CLOUD_JOB_ID --limit 1 | jq .data[0].id -r)
>> dbt-cloud run get-artifact --run-id $latest_run_id --path catalog.json -f catalog.json
>> dbt-cloud demo data-catalog -f catalog.json
##### ## ### ## ##
## ## ## ## ## ## ##
## ## ### ##### ### ## ## ### ##### ### ## ### ####
## ## # ## ## # ## ## # ## ## # ## ## ## ## ## ##
## ## #### ## #### ## #### ## #### ## ## ## # ##
## ## ## ## ## ## ## ## # ## ## ## ## ## ## ## ## ## #
## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##
##### ## ## ## ## ## #### ## ## ## ## ## ## ### ####
##
###
[?] Select node type to explore: source
> source
node
Thanks to Sean McIntyre for his initial work on triggering a dbt Cloud job using Python as proposed in this post on dbt Discourse. Thank you for sharing your work with the community!
FAQs
dbt Cloud command line interface (CLI)
We found that dbt-cloud-cli demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.