dbnd-mlflow

Overview

The dbnd-mlflow plugin allows storing mlflow metrics to DBND tracker together with duplicating them to the mlflow store.

Install

pip install dbnd-mlflow
# or
pip install databand[mlflow]

Config

[core]
# Databand store url should be defined
databand_url=http://localhost:8080

[mlflow_tracking]
# Enable tracking to Databand store
databand_tracking=True

# Optionally, define a URI for mlflow store,
# mlflow.get_tracking_uri() is used by default
; duplicate_tracking_to=http://mlflow-store/

Run example

You might need to install examples at first pip install dbnd-examples.

dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow

# or set configs manually
dbnd run dbnd_examples.tracking.tracking_mlflow.task_with_mflow --set-config mlflow_tracking.databand_tracking=True

Explanation

mlflow_example code

from dbnd import task
from mlflow import start_run, end_run
from mlflow import log_metric, log_param

@task
def mlflow_example():
    start_run()
    # params
    log_param("param1", randint(0, 100))
    log_param("param2", randint(0, 100))
    # metrics
    log_metric("foo1", random())
    log_metric("foo2", random())
    end_run()

Execution flow:

1. Run dbnd run mlflow_example --set-config mlflow_tracking.databand_tracking=True
2. dbnd creates a new dbnd context
3. dbnd_on_pre_init_context hook from dbnd_mlflow is triggered
   • a new uri is computed to be used by mlflow, e.g.:
     • dbnd://localhost:8080?duplicate_tracking_to=http%253A%252F%252Fmlflow-store%253A80%252F
   • the new uri is set to be used with mlflow.set_tracking_uri()
4. mlflow_example task starts:
   1. mlflow.start_run()
      1. mlflow reads entry_points for each installed package and finds:
         • "dbnd = dbnd_mlflow.tracking_store:get_dbnd_store",
         • "dbnd+s = dbnd_mlflow.tracking_store:get_dbnd_store",
         • "databand = dbnd_mlflow.tracking_store:get_dbnd_store",
         • "databand+s = dbnd_mlflow.tracking_store:get_dbnd_store",
      2. mlflow creates TrackingStoreClient using the new uri
      3. uri schema instructs to use dbnd_mlflow.tracking_store:get_dbnd_store
         • get_dbnd_store creates dbnd TrackingAPIClient
         • get_dbnd_store creates mlflow tracking store to duplicate tracking to
         • get_dbnd_store returns DatabandStore instance
   2. log_param()/log_metric()
      • calls to DatabandStore
        • calls to TrackingAPIClient
        • calls to mlflow tracking store to duplicate tracking to
   3. mlflow.end_run()
5. mlflow_example ends
6. dbnd_on_exit_context hook from dbnd_mlflow is triggered
   • restore original mlflow tracking uri