dalex: Responsible Machine Learning in Python
Unverified black box model is the path to the failure. Opaqueness leads to distrust. Distrust leads to ignoration. Ignoration leads to rejection.
The dalex package xrays any model and helps to explore and explain its behaviour, helps to understand how complex models are working.
The main Explainer object creates a wrapper around a predictive model. Wrapped models may then be explored and compared with a collection of model-level and predict-level explanations. Moreover, there are fairness methods and interactive exploration dashboards available to the user.
The philosophy behind dalex explanations is described in the Explanatory Model Analysis book.
The dalex package is available on PyPI and conda-forge.
pip install dalex -U
conda install -c conda-forge dalex
One can install optional dependencies for all additional features using pip install dalex[full].
The authors of the dalex package are:
arena modulefairness moduleaspect moduleaspect moduleWe welcome contributions: start by opening an issue on GitHub.
If you use dalex, please cite our JMLR paper:
@article{JMLR:v22:20-1473,
author = {Hubert Baniecki and
Wojciech Kretowicz and
Piotr Piatyszek and
Jakub Wisniewski and
Przemyslaw Biecek},
title = {dalex: Responsible Machine Learning
with Interactive Explainability and Fairness in Python},
journal = {Journal of Machine Learning Research},
year = {2021},
volume = {22},
number = {214},
pages = {1-7},
url = {http://jmlr.org/papers/v22/20-1473.html}
}
plotly dependency to <6.0.0, fixing compatibility issues with the new version, e.g. titlefont is now title_font (#573 contributed by @lionelkusch)numpy>=2.0.0 compatibility: replace instances of x.ptp() with np.ptp(x) and np.Inf with np.inf (#571)sample_weight to loss functions in model_parts() (variable importance) using weights from dx.Explainer (#563)shap_wrapper for shap==0.45.0python>=3.8, pandas>=1.5.0, numpy>=1.23.3 and add python==3.11 to CIkeras.src.models.sequential.Sequential to classes with a known predict_function; it should fix changes in keras==3.0.0 and tensorflow==2.16.0verbose in the predict method of tensorflow/keras models that changed in tensorflow>=2.9.0variable_splits (#558)predict_profile() when a DataFrame has MultiIndex in pandas>=1.3.0 (#550)norm() calculation in model_profile() from pi*sqrt(2) to sqrt(2*pi)prepare_numerical_categorical() and prepare_x() with pandas==2.1.0numeric_only in pandas.DataFrame.corr() in dalex.aspect.calculate_assoc_matrix()ZeroDivisionError to precision and recall functions (#532)calculate_depend_matrix() when there is a variable with only one value (#537)This release consists of mostly maintenance updates and, after a year, marks the Beta -> Stable release.
python>=3.6 to python>=3.7 (at this moment, both numpy and pandas depend on python>=3.8), and add python==3.10 to CIpandas>=1.2.5, numpy>=1.20.3 (#526), scipy>=1.6.3, plotly>=5.1.0, and tqdm>=4.61.2 due to errors with pandas (see tqdm/#1199)pd.Series.append() (#489)np.isnan causing error in dalex.fairness (#491)werkzeug server using a clearner and still supported API (#518)Arena (dalex/#408)facet_scales parameter to AP.plot and CP.plot, which allows to free the y-axis with facet_scales="free" (dalex/#469); consistent with R (DALEX/#468, ingredients/#140)AP and CP progress barsaspect module, which will focus on groups of dependent variables @krzyzinskim & @arturzolkowskiscipy>=1.5.4 dependencyVariableImportance.plot(split="variable") (#451)repr_html() didn't work for explanation objects before using the fit method (#449)Aspect object with the predict_triplot, model_triplot, predict_parts, model_parts, get_aspects methodsPredictTriplot, ModelTriplot, PredictAspectImportance, ModelAspectImportance objects with the plot methodresample, reweight, roc_pivot) into the fairness module (#432)set_options in Arena now takies option_category instead of plot_type (SHAPValues => ShapleyValues, FeatureImportance => VariableImportance) (#420)N parameter now properly sample rows from datapredict_function is found in Explainer (77ca90d)'spawn' (#412)metric_scores plot that made only one subgroup appear on y-axis (#416)ShapleyValuesImportance and ShapleyValuesDependence plots to Arena (#420)error instead of NaN when AUC is calculated on observations from one class only (#415)processes > 1 (#392), which means that the results of parallel computation will vary between v1.1.0 and previous versionsGroupFairnessX.plot(type='fairness_check') generates ticks according to the x-axis range (#409)GroupFainressRegression.plot(type='density') has a more readable hover - only for outliers (#409)BreakDown.plot() wrongly displayed the "+all factors" bar when max_vars < p (#401)GroupFairnessClassification.plot(type='metric_scores') did not handle NaN's (#399)fairness module. Added GroupFairnessRegression object, with the plot method having two types: fairness_check and density. Explainer.model_fairness method now depends on the model_type attribute. (#391)N parameter to the predict_parts method which is None by default (#402)epsilon is now an argument of the GroupFairnessClassification object (#397)yaxis in fairness_check plot (#376)np.float is depracated since numpy v1.20 (#384)ipython to test dependenciesThese are summed up in (#368):
dataset_level into model_explanations, instance_level into predict_explanations, _arena module into arena__dir__ method to define autocompletion in IPython environment - show only ['Explainer', 'Arena', 'fairness', 'datasets']plot method and result attribute to LimeExplanation (use lime.explanation.Explanation.as_pyplot_figure() and lime.explanation.Explanation.as_list())CeterisParibus.plot(variable_type='categorical') now has horizontal barplots - horizontal_spacing=None by default (varies on variable_type). Also, once again added the "dot" for observation value.predict_fn in predict_surrogate now uses predict_function (trying to make it work for more frameworks)y_hat/residuals was an int not float"-" sign to negative dropout losses in VariableImportance.plotgeom='bars' to AggregateProfiles.plot to force the categorical plotgeom='roc' and geom='lift' to ModelPerformance.plotcolorize from Explainerobject.py, move utils funcitons from checks.py to utils.py, etc.)h2o.estimators.* (#332)tensorflow.python.keras.engine.functional.Functional to the tensorflow listplotly dependency to >=4.12.0yhat, check_datacheck_if_empty_fields() used in loading the Explainer from a pickle file, since several checks were changedplot() method in GroupFairnessClassification as it omitted plotting a metric when NaN was present in metric ratios (result)dragons and HR datasets having , delimeter instead of ., which transformed numerical columns into categorical.ShapWrapper class (removed _repr_html_ method)y to be a pandas.DataFrame (converted)data, y to be a H2OFrame (converted)label parameter to all the relevant dx.Explainer methods, which overrides the default label in explanation's resultGradientExplainer for tf.keras.engine.sequential.Sequential, added proper warning when shap_explainer_type is None (#366)Explainerarena module, which adds the backend for Arena dashboard @piotrpiatyszekdx.Explainer methods (#350)
in model_parts it is {'permutational': 'variable_importance', 'feature_importance': 'variable_importance'},
in model_profile it is {'pdp': 'partial', 'ale': 'accumulated'}Arena object for dashboard backend. See https://github.com/ModelOriented/Arenafairness plot types: stacked, radar, performance_and_fairness, heatmap, ceteris_paribus_cutofffairness_check()fairness module, which will focus on bias detection, visualization and mitigation @jakwisnprecalculate=False and verbose=False (#340)model_fairness method to the Explainer, which performs fairness explanationGroupFairnessClassification object, with the plot method having two types: fairness_check and metric_scoresN=50000 argument to ResidualDiagnostics.plot, which samples observations from
the result parameter to omit performance issues when smooth=True (#341)tensorflow.python.keras.engine.sequential.Sequential and tensorflow.python.keras.engine.training.Model (#326)tqdm dependency to >=4.48.2, pandas dependency to >=1.1.2 and numpy dependency to >=1.18.4Explainer verbose messagesmodel_info parameter to be overwritten by the default valuesgroups was not of str type (#327)model_profile: variable_type='categorical' not working when user passed variables parameter (#329) +
the reverse order of bars in 'categorical' plots + (again) added variable_splits_type parameter to model_profile to specify how grid points
shall be calculated (#266) + allow for both 'quantile' and 'quantiles' types (alias)data and y to be None - added checks in Explainer methodstitle_x changed to y_title in CeterisParibus.plot and AggregatedProfiles.plot (#317)Explainer when predict_function returns an error or doesn't return numpy.ndarray (1d) (#325)pandas dependency to >=1.1.0ModelPerformance.plot now uses a drwhy color paletteunique method instead of np.unique in variable_splits (#293)v0.2.0 didn't export new datasetspredict_parts(type='shap') calculated wrong contributions
(#300)model_profile uses observation mean instead of profile mean in _yhat_ centeringmodel_profile and predict_profile plots
(#297)model_profile(type='accumulated') giving wrong results
(#302)type='shap_wrapper' to predict_parts and model_parts methods, which returns a new
ShapWrapper object. It contains the main result attribute (shapley_values) and the plot method
(force_plot and summary_plot respectively). These come from the shap packageExplainer.predict method now accepts numpy.ndarrayResidualDiagnostics object with a plot methodmodel_diagnostics method to the Explainer, which performs residual diagnosticspredict_surrogate method to the Explainer, which is a wrapper for the lime
tabular explanation from the lime packagemodel_surrogate method to the Explainer, which creates a basic surrogate decision tree
or linear model from the black-box model using the scikit-learn package_repr_html_ method to all of the explanation objects (it prints the result attribute)dalex.__version__Explainer methods when y is of wrong type
(#294)CeterisParibus.plot(variable_type='categorical') now allows for multiple observationsmodel_typetype to model_info in dump and dumps for R compatibility
(#303)ModelPerformance.result now has label as index_grid_ column in AggregatedProfiles.result and center only works with type=accumulatedPipeline._final_estimator to extract model_class of the actual modelmodel._estimator_type to extract model_type if possiblev0.1.9 had wrong _original_ column in predict_profilevertical_spacing acts as intended in VariableImportance.plot when split='variable'loss_function='auc' now uses loss_one_minus_auc as this should be a descending measuremodel_profile now properly passes the variables parameter to CeterisParibusvariables parameter in predict_profile now can also be a stringpx.express instead of core plotly to make model_profile and predict_profile plots;
thus, enhance performance and scalabilityverbose parameter where tqdm is used to verbose progress barloss_one_minus_auc function that can be used with loss_function='1-auc' in model_partsapartments, dragons and hrcolor, opacity, title_x parameters to model_profile and predict_profile plots (#236),
changed tooltips and legends (#262)geom='profiles' parameter to model_profile plot and raw_profiles attribute to AggregatedProfilesvariable_splits_type parameter to predict_profile to specify how grid points
shall be calculated (#266)variable_splits_with_obs parameter to predict_profile function to extend split points with observation
variable values (#269)variable_splits parameter to model_profileloss_function for classification and regression (#248)proba yhats now get model_type='classification' if it's not specifiedpredict_profile and model_profile (see variable_splits_type parameter)new_observation to variable_splits in predict_profile (see variable_splits_with_obs parameter)N=1000 in model_parts and N=300 in model_profile to comply with the R versionkeep_raw_permutation is now set to False instead of None in model_partsintercept parameter in model_profile is now named centerrandom_state parameter for predict_parts(type='shap') and model_profile for reproducible calculationsrandom_state parameter in model_partsmodel_profile, model_parts, predict_profile and predict_parts(type='shap'), through the processes parameteraccumulated and conditional types in model_profilenp.issubdtype() to cover more types; e.g. it caused errors with string typeCeterisParibus to fix variable dtypes and
later allow for a concatenation without the dtype conversionvariables parameter now can be a single str valuepredict_parts, model_parts (#245)variable_splits parameter now works correctly in predict_profileAggregatedProfiles.plot (#234)Explainer messagesinstance_levelAggregatedProfiles.plot now works with groupsN in model_profile can be set to None, to select all observationsgroups and variable parameters in model_profile can be: str, list, numpy.ndarray, pandas.Seriescheck_label returned only a first letterall_variables to str in
prepare_all_variables, which caused an error in model_profile (#214)short_name encoding in fifa dataset (utf8->ascii)scipy dependencyloss_root_mean_square in model parts changed to rmsenew_observation in BreakDown, Shap, CeterisParibus now work for multiple inputs
(#207)CeterisParibus.fit and CeterisParibus.plot now work for
more types of new_observation.index, but won't work for a bolean type
(#211)xgboost package compatibility (#188)model_class parameter to Explainer to handle wrapped
modelsExaplainer attribute model_info remembers if parameters are defaultvariable_groups parameter now works correctly in model_partsExplainer: model_type, model_info,
colorizemodel_parts documentation is updatedshow parameter in plot methods that (if False) returns
plotly Figure (#190)load_fifa() function which loads the preprocessed players_20
datasetCeterisParibus.plot tooltipExplainer.residual method which uses
residual_function to calculate residualsdump and dumps methods for saving Explainer in a binary form;
load and loads methods for loading Explainer from binary formExplainer constructor verbose textB:=B+1 - Shap now stores average results as B=0 and path results as B=1,2,...Explainer.model_performance method uses self.model_type when
model_type is NoneBreakDown and Shap are now rounded to 4 significant
places (#180)Shap by default uses path='average', sign column is
properly updated and bars in plot are sorted by abs(contribution)dalex packageExplainer object with predict, predict_parts, predict_profile,
model_performance, model_parts and model_profile methodsBreakDown, Shap, CeterisParibus, ModelPerformance,
VariableImportance and AggregatedProfiles objects with a plot methodload_titanic() function which loads the titanic_imputed datasetFAQs
Responsible Machine Learning in Python
We found that dalex demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Bybit's $1.46B hack by North Korea's Lazarus Group pushes 2025 crypto losses to $1.6B in just two months, already surpassing all of 2024's $1.49B total.
Security News
OpenSSF has published OSPS Baseline, an initiative designed to establish a minimum set of security-related best practices for open source software projects.
Security News
Michigan TypeScript founder Dimitri Mitropoulos implements WebAssembly runtime in TypeScript types, enabling Doom to run after processing 177 terabytes of type definitions.