faster-coco-eval

Faster-COCO-Eval

Why should you replace pycocotools with faster-coco-eval?

Aspect	pycocotools	faster-coco-eval
Support & Development	Outdated and not actively maintained. Issues and incompatibilities arise with new releases.	Actively maintained, continuously evolving, and regularly updated with new features and bug fixes.
Transparency & Reliability	Lacks comprehensive testing, making updates risky and results less predictable.	Emphasizes extensive test coverage and code quality, ensuring trustworthy and reliable results.
Performance	Significantly slower, especially on large datasets or distributed workloads.	Several times faster due to C++ optimizations and modern algorithms.
Functionality	Limited to basic COCO format evaluation.	Offers extended metrics, support for new IoU types, compatibility with more datasets (e.g., CrowdPose, LVIS), advanced visualizations, and seamless integration with PyTorch/TorchVision.

By choosing faster-coco-eval, you benefit from:

• Reliability and confidence in your results
• High processing speed
• Modern functionality and support for new tasks
• An active community and prompt response to your requests

Switch to faster-coco-eval and experience a new standard in working with COCO annotations!

Install

Basic implementation identical to pycocotools

pip install faster-coco-eval

Additional visualization options

Only 1 additional package needed opencv-python-headless

pip install faster-coco-eval[extra]

Conda install

conda install conda-forge::faster-coco-eval

Basic usage

import faster_coco_eval

# Replace pycocotools with faster_coco_eval
faster_coco_eval.init_as_pycocotools()

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval

anno = COCO(str(anno_json))  # init annotations api
pred = anno.loadRes(str(pred_json))  # init predictions api (must pass string, not Path)

val = COCOeval(anno, pred, "bbox")
val.evaluate()
val.accumulate()
val.summarize()

Faster-COCO-Eval base

This package wraps a facebook C++ implementation of COCO-eval operations found in the pycocotools package. This implementation greatly speeds up the evaluation time for coco's AP metrics, especially when dealing with a high number of instances in an image.

Comparison

For our use case with a test dataset of 5000 images from the coco val dataset. Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.

Visualization of testing colab_example.ipynb available in directory examples/comparison

• mmdet example
• ultralytics example

Summary for 5000 imgs

Type	faster-coco-eval	pycocotools	Profit
bbox	5.812	22.72	3.909
segm	7.413	24.434	3.296

Feautures

This library provides not only validation functions, but also error visualization functions. Including visualization of errors in the image. You can study in more detail in the examples and Wiki.

Usage

Code examples for using the library are available on the Wiki

Examples

• Eval example
• Curve example
• LVIS example
• Crowdpose example
• Custom keypoints example
• showAnns example

Update history

Available via link history.md

Star History

License

The original module was licensed with apache 2, I will continue with the same license. Distributed under the apache version 2.0 license, see license for more information.

Citation

If you use this benchmark in your research, please cite this project.

@article{faster-coco-eval,
  title   = {{Faster-COCO-Eval}: Faster interpretation of the original COCOEval},
  author  = {MiXaiLL76},
  year    = {2024}
}