Bamboo is a library intended to facilitate the process of creating extract, transform, and load (ETL) data pipelines. Bamboo includes many features such as the ability to download and cache HTTP assets, copy files from remote servers, run commands on remote servers, handle zipped archives, perform bulk database ingests and more.
pip install bamboo-lib
If you will need to use the distributed locking functionality, you will need to install some additional software. Below are the instructions for macOS
brew install libmemcached
pip install pylibmc --install-option="--with-libmemcached=/usr/local/Cellar/libmemcached/1.0.18_2"
pip install sherlock
To run the tests, simply run:
pytest
Alternatively, if you would like to display all log/print statements run:
pytest -s
To change the default folder where Bamboo will store HTTP downloads, set the BAMBOO_DOWNLOAD_FOLDER environment variable. By default, downloads will go to /tmp.
To change the default logger settings, set BAMBOO_LOGGER_CONF to point to a logging configuration file (see example in example/logging.conf).
If you are running High Sierra or later and get an error like:
objc[30911]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug
Try setting:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
FAQs
Python ETL library
We found that bamboo-lib demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.