Docling-eval

Evaluate Docling on various datasets.
Features
Evaluate docling on various datasets. You can use the cli
terminal %> docling-eval --help
Usage: docling_eval [OPTIONS] COMMAND [ARGS]...
Docling Evaluation CLI for benchmarking document processing tasks.
╭─ Options ────────────────────────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────╮
│ create Create both ground truth and evaluation datasets in one step. │
│ create-eval Create evaluation dataset from existing ground truth. │
│ create-gt Create ground truth dataset only. │
│ evaluate Evaluate predictions against ground truth. │
│ visualize Visualize evaluation results. │
╰──────────────────────────────────────────────────────────────────────────────────────╯
Benchmarks
- General
- DP-Bench benchmarks: Text, layout, reading order and table structure evaluation on the DP-Bench dataset.
- OmniDocBench benchmarks: Text, layout, reading order and table structure evaluation on the OmniDocBench dataset.
- Layout
- Table-Structure
On our list for next benchmarks:
Contributing
Please read Contributing to Docling for details.
License
The Docling codebase is under MIT license.
For individual model usage, please refer to the model licenses found in the original packages.
IBM ❤️ Open Source AI
Docling-eval has been brought to you by IBM.