quantile-data-kit

Quantile Data Kit 🔍

Publish to pypi

How to deploy a new version of the QDK?

• Update the package version in setup.py.
• Run the Makefile make publish

Components

There are four types of base components in the QDK.

• LoadComponent. Takes nothing as input and outputs a DataFrame.
• TransformComponent. Takes a DataFrame as input and outputs a DataFrame.
• TrainingComponent. Takes data and a model as input and outputs a trained model.
• InferenceComponent. Takes data and a model as input and ouputs prediction data.

Adding a new component?

Adding a new component to the QDK requires the following steps:

• Type of component: Decide which type of the four components above you are adding.
• Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g. qdk/loader) a new Python file that inherits from the parent component. In this file you can optionally overwrite input_defs, output_defs and config_schema. When adding a new component, you are required to add a classmethod with the same name as the compute_function attribute on the parent class. The keys in the config_schema are injected into the parameters of the compute function. Lastly, you need to import the new component to qdk/__init__.py. This allows you to import it from top-level.
• Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using pytest. The tests can be added at qdk/tests. Reminder to prefix the folder, files and functions with test_. One is able to test the components using either VScode testing or the terminal (e.g. with pytest -s qdk/tests/test_loaders).