hydraflow

Hydraflow

Overview

Hydraflow is a library designed to seamlessly integrate Hydra and MLflow, making it easier to manage and track machine learning experiments. By combining the flexibility of Hydra's configuration management with the robust experiment tracking capabilities of MLflow, Hydraflow provides a comprehensive solution for managing complex machine learning workflows.

Key Features

• Configuration Management: Utilize Hydra's advanced configuration management to handle complex parameter sweeps and experiment setups.
• Experiment Tracking: Leverage MLflow's tracking capabilities to log parameters, metrics, and artifacts for each run.
• Artifact Management: Automatically log and manage artifacts, such as model checkpoints and configuration files, with MLflow.
• Seamless Integration: Easily integrate Hydra and MLflow in your machine learning projects with minimal setup.

Installation

You can install Hydraflow via pip:

pip install hydraflow

Getting Started

Here is a simple example to get you started with Hydraflow:

import hydra
import hydraflow
import mlflow
from dataclasses import dataclass
from hydra.core.config_store import ConfigStore
from pathlib import Path

@dataclass
class MySQLConfig:
    host: str = "localhost"
    port: int = 3306

cs = ConfigStore.instance()
cs.store(name="config", node=MySQLConfig)

@hydra.main(version_base=None, config_name="config")
def my_app(cfg: MySQLConfig) -> None:
    # Set experiment by Hydra job name.
    hydraflow.set_experiment()

    # Automatically log Hydra config as params.
    with hydraflow.start_run():
        # Your app code below.

        with hydraflow.watch(callback):
            # Watch files in the MLflow artifact directory.
            # You can update metrics or log other artifacts
            # according to the watched files in your callback
            # function.
            pass

# Your callback function here.
def callback(file: Path) -> None:
    pass

if __name__ == "__main__":
    my_app()