dkist-processing-test
Advanced tools
Example instrument code used by the DKIST science data processing pipelines to test processing infrastructure
|codecov|
The dkist-processing-test library serves as an example implementation of a Tasks and Workflows using the
dkist-processing-core <https://pypi.org/project/dkist-processing-core/>_ framework and
dkist-processing-common <https://pypi.org/project/dkist-processing-common/>_ Tasks.
The recommended project structure is to separate tasks and workflows into separate packages.
Artifacts are built through bitbucket pipelines <bitbucket-pipelines.yml>_
The pipeline can be used in other repos with a modification of the package and artifact locations to use the names relevant to the target repo.
e.g. dkist-processing-test -> dkist-processing-vbi and dkist_processing_test -> dkist_processing_vbi
Deployment is done with turtlebot <https://bitbucket.org/dkistdc/turtlebot/src/master/>_ and follows
the process detailed in dkist-processing-core <https://pypi.org/project/dkist-processing-core/>_
Only those specified by dkist-processing-core <https://pypi.org/project/dkist-processing-core/>_ and dkist-processing-common <https://pypi.org/project/dkist-processing-common/>_
.. code-block:: bash
git clone git@bitbucket.org:dkistdc/dkist-processing-test.git
cd dkist-processing-test
pre-commit install
pip install -e .[test]
pytest -v --cov dkist_processing_test
When a new release is ready to be built the following steps need to be taken:
A new "frozen" extra is generated by the dkist-dev-tools <https://bitbucket.org/dkistdc/dkist-dev-tools/src/main/>_
package. If you don't have dkist-dev-tools installed please follow the directions from that repo.
To freeze dependencies run
.. code-block:: bash
ddt freeze vX.Y.Z[rcK]
Where "vX.Y.Z[rcK]" is the version about to be released.
Once all commits are in place add a git tag that will define the released version, then push the tags up to Bitbucket:
.. code-block:: bash
git tag vX.Y.Z[rcK]
git push --tags origin BRANCH
In the case of an rc, BRANCH will likely be your development branch. For full releases BRANCH should be "main".
.. |codecov| image:: https://codecov.io/bb/dkistdc/dkist-processing-test/graph/badge.svg?token=U004CWS46G :target: https://codecov.io/bb/dkistdc/dkist-processing-test
FAQs
Example instrument code used by the DKIST science data processing pipelines to test processing infrastructure
We found that dkist-processing-test demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
A survey of 500 cybersecurity pros reveals high pay isn't enough—lack of growth and flexibility is driving attrition and risking organizational security.
Product
Socket, the leader in open source security, is now available on Google Cloud Marketplace for simplified procurement and enhanced protection against supply chain attacks.
Security News
Corepack will be phased out from future Node.js releases following a TSC vote.
