faster-coco-eval

Faster-COCO-Eval

Disclaimer

I often use this project, but I saw it abandoned and without a public repository on github. Also, part of the project remained unfinished for a long time. I implemented some of the author's ideas and decided to make the results publicly available.

Install

Basic implementation identical to pycocotools

pip install faster-coco-eval

Additional visualization options

Only 1 additional package needed opencv-python-headless

pip install faster-coco-eval[extra]

Basic usage

import faster_coco_eval

# Replace pycocotools with faster_coco_eval
faster_coco_eval.init_as_pycocotools()

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval

anno = COCO(str(anno_json))  # init annotations api
pred = anno.loadRes(str(pred_json))  # init predictions api (must pass string, not Path)

val = COCOeval(anno, pred, "bbox")
val.evaluate()
val.accumulate()
val.summarize()

Faster-COCO-Eval base

This package wraps a facebook C++ implementation of COCO-eval operations found in the pycocotools package. This implementation greatly speeds up the evaluation time for coco's AP metrics, especially when dealing with a high number of instances in an image.

Comparison

For our use case with a test dataset of 5000 images from the coco val dataset. Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.

Visualization of testing colab_example.ipynb available in directory examples/comparison

• mmdet example
• ultralytics example

Summary for 5000 imgs

Type	faster-coco-eval	pycocotools	Profit
bbox	5.812	22.72	3.909
segm	7.413	24.434	3.296

Feautures

This library provides not only validation functions, but also error visualization functions. Including visualization of errors in the image. You can study in more detail in the examples and Wiki.

Usage

Code examples for using the library are available on the Wiki

Examples

• Eval example
• Curve example
• CED Keypoint example
• LVIS example

Update history

Available via link history.md

Star History

License

The original module was licensed with apache 2, I will continue with the same license. Distributed under the apache version 2.0 license, see license for more information.

Citation

If you use this benchmark in your research, please cite this project.

@article{faster-coco-eval,
  title   = {{Faster-COCO-Eval}: Faster interpretation of the original COCOEval},
  author  = {MiXaiLL76},
  year    = {2024}
}