Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
The Databricks Lakehouse provides one simple platform to unify all your data, analytics and AI workloads.
The dbt-databricks
adapter contains all of the code enabling dbt to work with Databricks. This adapter is based off the amazing work done in dbt-spark. Some key features include:
MERGE
as the the default incremental materialization strategy.If you are developing a dbt project on Databricks, we recommend using dbt-databricks
for the reasons noted above.
dbt-spark
is an actively developed adapter which works with Databricks as well as Apache Spark anywhere it is hosted e.g. on AWS EMR.
Install using pip:
pip install dbt-databricks
Upgrade to the latest version
pip install --upgrade dbt-databricks
your_profile_name:
target: dev
outputs:
dev:
type: databricks
catalog: [optional catalog name, if you are using Unity Catalog, only available in dbt-databricks>=1.1.1]
schema: [database/schema name]
host: [your.databrickshost.com]
http_path: [/sql/your/http/path]
token: [dapiXXXXXXXXXXXXXXXXXXXXXXX]
These following quick starts will get you up and running with the dbt-databricks
adapter:
The dbt-databricks
adapter has been tested:
Databricks SQL
and Databricks runtime releases 9.1 LTS
and later.You can override the compute used for a specific Python model by setting the http_path
property in model configuration. This can be useful if, for example, you want to run a Python model on an All Purpose cluster, while running SQL models on a SQL Warehouse. Note that this capability is only available for Python models.
def model(dbt, session):
dbt.config(
http_path="sql/protocolv1/..."
)
FAQs
The Databricks adapter plugin for dbt
We found that dbt-databricks demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.