DataEval provides a simple interface to characterize image data and its impact on model performance across classification and object-detection tasks
DataEval focuses on characterizing image data and its impact on model performance across Classification and object-detection tasks.
Model-agnostic metrics that bound real-world performance
Model-specific metrics that guide model selection and training
Metrics for post-deployment monitoring of data with bounds on model performance to guide retraining
You can install DataEval directly from pypi.org using the following command. The optional dependencies of DataEval are torch, tensorflow and all. Using torch enables Sufficiency metrics, and tensorflow enables OOD Detection.
pip install dataeval[all]
DataEval can be installed in a Conda/Mamba environment using the provided environment.yaml file. As some dependencies
are installed from the pytorch channel, the channel is specified in the below example.
micromamba create -f environment\environment.yaml -c pytorch
To install DataEval from source locally on Ubuntu, you will need git-lfs to download larger, binary source files and poetry for project dependency management.
sudo apt-get install git-lfs
pip install poetry
Pull the source down and change to the DataEval project directory.
git clone https://github.com/aria-ml/dataeval.git
cd dataeval
Install DataEval with optional dependencies for development.
poetry install --all-extras --with dev
Now that DataEval is installed, you can run commands in the poetry virtual environment by prefixing shell commands with poetry run, or activate the virtual environment directly in the shell.
poetry shell
For more ideas on getting started using DataEval in your workflow, additional information and tutorials are in our Sphinx documentation hosted on Read the Docs.
This project uses code from the Alibi-Detect python library developed by SeldonIO. Additional documentation from the developers are also available here.
FAQs
DataEval provides a simple interface to characterize image data and its impact on model performance across classification and object-detection tasks
We found that dataeval demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.