AutoPrognosis - A system for automating the design of predictive modeling pipelines tailored for clinical prognosis.
![AutoPrognosis image](https://github.com/vanderschaarlab/autoprognosis/raw/main/docs/arch.png)
:key: Features
- :rocket: Automatically learns ensembles of pipelines for classification, regression or survival analysis tasks.
- :cyclone: Easy to extend pluginable architecture.
- :fire: Interpretability and uncertainty quantification tools.
- :adhesive_bandage: Data imputation using HyperImpute.
- :zap: Build demonstrators using Streamlit.
- :notebook: Python and R tutorials available.
- :book: Read the docs
:rocket: Installation
Using pip
The library can be installed from PyPI using
$ pip install autoprognosis
or from source, using
$ pip install .
Redis (Optional, but recommended)
AutoPrognosis can use Redis as a backend to improve the performance and quality of the searches.
For that, install the redis-server package following the steps described on the official site.
Environment variables
The library can be configured from a set of environment variables.
Variable | Description |
---|
N_OPT_JOBS | Number of cores to use for hyperparameter search. Default : 1 |
N_LEARNER_JOBS | Number of cores to use by inidividual learners. Default: all cpus |
REDIS_HOST | IP address for the Redis database. Default 127.0.0.1 |
REDIS_PORT | Redis port. Default: 6379 |
Example: export N_OPT_JOBS = 2
to use 2 cores for hyperparam search.
:boom: Sample Usage
Advanced Python tutorials can be found in the Python tutorials section.
R examples can be found in the R tutorials section.
List the available classifiers
from autoprognosis.plugins.prediction.classifiers import Classifiers
print(Classifiers().list_available())
Create a study for classifiers
from sklearn.datasets import load_breast_cancer
from autoprognosis.studies.classifiers import ClassifierStudy
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_estimator
X, Y = load_breast_cancer(return_X_y=True, as_frame=True)
df = X.copy()
df["target"] = Y
study_name = "example"
study = ClassifierStudy(
study_name=study_name,
dataset=df,
target="target",
)
model = study.fit()
model.predict_proba(X)
(Advanced) Customize the study for classifiers
from pathlib import Path
from sklearn.datasets import load_breast_cancer
from autoprognosis.studies.classifiers import ClassifierStudy
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_estimator
X, Y = load_breast_cancer(return_X_y=True, as_frame=True)
df = X.copy()
df["target"] = Y
workspace = Path("workspace")
study_name = "example"
study = ClassifierStudy(
study_name=study_name,
dataset=df,
target="target",
num_iter=100,
timeout=60,
classifiers=["logistic_regression", "lda", "qda"],
workspace=workspace,
)
study.run()
output = workspace / study_name / "model.p"
model = load_model_from_file(output)
metrics = evaluate_estimator(model, X, Y)
print(f"model {model.name()} -> {metrics['str']}")
model.fit(X, Y)
model.predict_proba(X)
List the available regressors
from autoprognosis.plugins.prediction.regression import Regression
print(Regression().list_available())
Create a Regression study
import pandas as pd
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_regression
from autoprognosis.studies.regression import RegressionStudy
df = pd.read_csv(
"https://archive.ics.uci.edu/ml/machine-learning-databases/00291/airfoil_self_noise.dat",
header=None,
sep="\\t",
)
last_col = df.columns[-1]
y = df[last_col]
X = df.drop(columns=[last_col])
df = X.copy()
df["target"] = y
study_name="regression_example"
study = RegressionStudy(
study_name=study_name,
dataset=df,
target="target",
)
model = study.fit()
model.predict(X)
Advanced Customize the Regression study
from pathlib import Path
import pandas as pd
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_regression
from autoprognosis.studies.regression import RegressionStudy
df = pd.read_csv(
"https://archive.ics.uci.edu/ml/machine-learning-databases/00291/airfoil_self_noise.dat",
header=None,
sep="\\t",
)
last_col = df.columns[-1]
y = df[last_col]
X = df.drop(columns=[last_col])
df = X.copy()
df["target"] = y
workspace = Path("workspace")
workspace.mkdir(parents=True, exist_ok=True)
study_name="regression_example"
study = RegressionStudy(
study_name=study_name,
dataset=df,
target="target",
num_iter=10,
num_study_iter=2,
timeout=50,
regressors=["linear_regression", "xgboost_regressor"],
workspace=workspace,
)
study.run()
output = workspace / study_name / "model.p"
model = load_model_from_file(output)
metrics = evaluate_regression(model, X, y)
print(f"Model {model.name()} score: {metrics['str']}")
model.fit(X, y)
model.predict(X)
List available survival analysis estimators
from autoprognosis.plugins.prediction.risk_estimation import RiskEstimation
print(RiskEstimation().list_available())
Create a Survival analysis study
import numpy as np
from pycox import datasets
from autoprognosis.studies.risk_estimation import RiskEstimationStudy
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_survival_estimator
df = datasets.gbsg.read_df()
df = df[df["duration"] > 0]
X = df.drop(columns = ["duration"])
T = df["duration"]
Y = df["event"]
eval_time_horizons = np.linspace(T.min(), T.max(), 5)[1:-1]
study_name = "example_risks"
study = RiskEstimationStudy(
study_name=study_name,
dataset=df,
target="event",
time_to_event="duration",
time_horizons=eval_time_horizons,
)
model = study.fit()
model.predict(X, eval_time_horizons)
Advanced Customize the Survival analysis study
import os
from pathlib import Path
import numpy as np
from pycox import datasets
from autoprognosis.studies.risk_estimation import RiskEstimationStudy
from autoprognosis.utils.serialization import load_model_from_file
from autoprognosis.utils.tester import evaluate_survival_estimator
df = datasets.gbsg.read_df()
df = df[df["duration"] > 0]
X = df.drop(columns = ["duration"])
T = df["duration"]
Y = df["event"]
eval_time_horizons = np.linspace(T.min(), T.max(), 5)[1:-1]
workspace = Path("workspace")
study_name = "example_risks"
study = RiskEstimationStudy(
study_name=study_name,
dataset=df,
target="event",
time_to_event="duration",
time_horizons=eval_time_horizons,
num_iter=10,
num_study_iter=1,
timeout=10,
risk_estimators=["cox_ph", "survival_xgboost"],
score_threshold=0.5,
workspace=workspace,
)
study.run()
output = workspace / study_name / "model.p"
model = load_model_from_file(output)
metrics = evaluate_survival_estimator(model, X, T, Y, eval_time_horizons)
print(f"Model {model.name()} score: {metrics['str']}")
model.fit(X, T, Y)
model.predict(X, eval_time_horizons)
:high_brightness: Tutorials
Plugins
AutoML
Building a demonstrator
:zap: Plugins
Imputation methods
from autoprognosis.plugins.imputers import Imputers
imputer = Imputers().get(<NAME>)
Preprocessing methods
from autoprognosis.plugins.preprocessors import Preprocessors
preprocessor = Preprocessors().get(<NAME>)
Name | Description |
---|
maxabs_scaler | Scale each feature by its maximum absolute value. MaxAbsScaler |
scaler | Standardize features by removing the mean and scaling to unit variance. - StandardScaler |
feature_normalizer | Normalize samples individually to unit norm. Normalizer |
normal_transform | Transform features using quantiles information.QuantileTransformer |
uniform_transform | Transform features using quantiles information.QuantileTransformer |
minmax_scaler | Transform features by scaling each feature to a given range.MinMaxScaler |
Classification
from autoprognosis.plugins.prediction.classifiers import Classifiers
classifier = Classifiers().get(<NAME>)
Survival Analysis
from autoprognosis.plugins.prediction.risk_estimation import RiskEstimation
predictor = RiskEstimation().get(<NAME>)
Regression
from autoprognosis.plugins.prediction.regression import Regression
regressor = Regression().get(<NAME>)
Explainers
from autoprognosis.plugins.explainers import Explainers
explainer = Explainers().get(<NAME>)
Uncertainty
from autoprognosis.plugins.uncertainty import UncertaintyQuantification
model = UncertaintyQuantification().get(<NAME>)
Name | Description |
---|
cohort_explainer | |
conformal_prediction | |
jackknife | |
:hammer: Test
After installing the library, the tests can be executed using pytest
$ pip install .[testing]
$ pytest -vxs -m "not slow"
Citing
If you use this code, please cite the associated paper:
@misc{https://doi.org/10.48550/arxiv.2210.12090,
doi = {10.48550/ARXIV.2210.12090},
url = {https://arxiv.org/abs/2210.12090},
author = {Imrie, Fergus and Cebere, Bogdan and McKinney, Eoin F. and van der Schaar, Mihaela},
keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
References
- AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning
- Prognostication and Risk Factors for Cystic Fibrosis via Automated Machine Learning
- Cardiovascular Disease Risk Prediction using Automated Machine Learning: A Prospective Study of 423,604 UK Biobank Participants