Recce is data validation toolkit for pull request (PR) review in dbt projects. Get enhanced visibility into how your team’s dbt modeling changes impact data by comparing your dev branch with stable production data. Run manual data checks during development, and automate checks in CI for PR review.
Get up and running quickly by prepping your dev and prod environments. The key is building prod into the target-base folder to use as the base for the data comparison.
# Build prod and generate dbt docs into ./target-base
dbt seed --target prod
dbt run --target prod
dbt docs generate --target prod --target-path ./target-base
# Switch to your dev branch
git switch my-awesome-branch
# build your dev environment
dbt seed
dbt run
dbt docs generate
# Start a Recce Instance
recce server
Follow our 5-minute Jaffle Shop tutorial to try it out for yourself.
recce server launches a web UI that shows you the area of your lineage that is impacted by the branch changes.
Recce Cloud) Automatically sync Check status between Recce InstancesRecce Cloud) Block PR merging until all Recce Checks have been approvedRead more about using Recce for Impact Assessment on the Recce blog.
We provide three online Recce demos (based on Jaffle Shop), each is related to a specific pull request. Use these demos to inspect the data impact caused by the modeling changes in the PR.
For each demo, review the following:
RecceThis will enable you to validate if the intention of the PR has been successfully implemented without unintended impact.
[!TIP] Don't forget to click the Checks tab to view the Recce Checklist, and perform your own Checks for further investigation.
This pull request adjusts the logic for how customer lifetime value is calculated:
This pull request performs some refactoring on the customers model by turning two CTEs into intermediate models, enhancing readability and maintainability:
This pull request introduces a new Rounding Effect Analysis feature, aimed at analyzing and reporting the impacts of rounding in our data processing.
This pull request enhances the existing customers model by categorizing whether a customer has placed an order as part of a promotion.
Reccedbt has brought many software best practices to data projects, such as:
Even so, 'bad merges' still happen and erroneous data and silent errors make their way into prod data. As self-serve analytics opens dbt projects to many roles, and the size of dbt projects increase, the job of reviewing data modeling changes is even more critical.
The only way to understand the impact of code changes on data is to compare the data before-and-after the changes.
Recce provides a data review environment for data teams to check their work during development, and then again as part of PR review. The suite of tools and diffs in Recce are specifically geared towards surfacing, understanding, and recording data impact from code changes.
Lineage Diff is the main interface to Recce and shows which nodes in the lineage have been added, removed, or modified.
Advanced Diffs provide high level statistics about data change:
Query Diff compares the results of any ad-hoc query, and supports the use of dbt macros.
The checklist provides a way to record the results of your data validation process.
Recce Cloud) Sync checks and check results across Recce instancesRecce Cloud) Block PR merging until checks have been approvedRecce?Recce is useful for validating your own work or the work of others, and can also be used to share data impact with non-technical stakeholders to approve data checks.
Recce to ensure the structural integrity of the data and understand the scope of impact before merging.Recce to self-review and understand how data modeling changes have changed the data.Recce to sign-off on data after updates have been madeRecceThe Recce Documentation covers everything you need to get started.
We’d advise first following the 5-minute tutorial that uses Jaffle Shop and then trying out Recce in your own project.
For advice on best practices in preparing dbt environments to enable effective PR review, check out Best Practices for Preparing Environments.
Recce Cloud provides a backbone of supporting services that make Recce usage more suitable for teams reviewing multiple pull requests.
With Recce Cloud:
Recce Instances can be launched directly from a PRRecce InstancesRecce Cloud is currently in early-access private beta.
To find out how you can get access please book an appointment for a short meeting.
Recce consists of a local server application that you run on your own device or compute services.
Recce happen either in your data warehouse, or in the browser itself.Recce does not store your data.For Recce Cloud users:
Recce state file is storedon Recce Cloud. This file is encrypted before transmission.Here's where you can get in touch with the Recce team and find support:
If you believe you have found a bug, or there is some missing functionality in Recce, please open a GitHub Issue.
You can follow along with news about Recce and blogs from our team in the following places:
FAQs
Environment diff tool for dbt
We found that recce demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Malicious Go packages are impersonating popular libraries to install hidden loader malware on Linux and macOS, targeting developers with obfuscated payloads.
Security News
Bybit's $1.46B hack by North Korea's Lazarus Group pushes 2025 crypto losses to $1.6B in just two months, already surpassing all of 2024's $1.49B total.
Security News
OpenSSF has published OSPS Baseline, an initiative designed to establish a minimum set of security-related best practices for open source software projects.