celery-prometheus

Celery Prometheus

This module expose the Prometheus HTTP server to expose metrics of your Celery backends.

To install celery-prometheus with pip, use the command:

pip install celery-prometheus

With Poetry:

poetry add celery-prometheus

Usage

To setup celery-prometheus to your backend, simply call the method add_prometheus_option after the init of the Celery object.

Example:

from celery import Celery
from celery_prometheus import add_prometheus_option

app = Celery()
add_prometheus_option(app)

# Rest of your code ...

Before starting your backend, you will need to expose the PROMETHEUS_MULTIPROC_DIR environment variable to indicate which folder the Prometheus Client will use to store the metrics (see Multiprocess Mode (E.g. Gunicorn) of the Promehteus Client documentation).

To start and expose the Prometheus HTTP Server, you need to use the --prometheus-collector-addr argument when starting your Celery backend:

export PROMETHEUS_MULTIPROC_DIR=/var/cache/my_celery_app
celery worker -A my_celery_backend.backend --prometheus-collector-addr 0.0.0.0:6543

Now that your backend is started, you can configure your Prometheus scrappers to scrappe your Celery backend.

Contributions

This project is open to external contributions. Feel free to submit us a Pull request if you want to contribute and improve with us this project.

In order to maintain an overall good code quality, this project use the following tools:

• Black
• Isort
• Flake8

Linting and formatting tools are configured to match the current default rules of Black.

We also use Mypy as a static type checker.

Please ensure to run these tools before commiting and submiting a Pull request. In case one of these mentionned tools report an error, the CI will automatically fail.

If you're making your first contribution to this project, please add your name to the contributors list.

License

This project is released by Gandi.net tech team under the BSD-3 license.