SPETLR-tools is a library that provides a set of tools for working with Databricks Lakehouses. These tools include test fixtures and development utilities that are not part of the runtime tools in SPETLR.
Visit the official SPETLR webpage: https://spetlr.com/
SPETLR-tools is designed to support SPETLR in various scenarios, including:
test_requirements.txtInstall SPETLR from PyPI:
pip install spetlr-tools
To prepare for development, please install following additional requirements:
pip install -r test_requirements.txtThen install the package locally:
python setup.py develop
After installing the dev-requirements, execute tests by running:
pytest tests
These tests are located in the ./tests/unit folder and only require a Python interpreter. Pull requests will not be accepted if these tests do not pass. If you add new features, please include corresponding tests.
During the pre-integration workflow (.gitub/workflows/pre-integration.yml) spetlr-tool supported CLI are (should) be tested.
Feel free to contribute to SPETLR-tools. Any contributions are appreciated - not only new features, but also if you find a way to improve SPETLR-tools.
If you have a suggestion that can enhance SPETLR-tools, please fork the repository and create a pull request. Alternatively, you can open an issue with the "enhancement" tag.
git checkout -b feature/NewSPETLRToolsFeature)git commit -m 'Add some SEPTLRToolsFeature')git push origin feature/NewSPETLRToolsFeature)
Releases to PyPI is an Github Action which needs to be manually triggered.
For any inquiries, please use the SPETLR Discord Server.
FAQs
Supplements to the python SPark ETL libRary (SPETLR) for Databricks.
We found that spetlr-tools demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.