SMART Model Card
A Python library for generating standardized AI/ML model documentation cards with OMOP Common Data Model integration for healthcare applications.
Overview
SMART Model Card provides tools for creating structured model documentation following healthcare AI/ML best practices. The library includes:
- Interactive CLI wizard with validation
- OMOP CDM integration via OHDSI WebAPI
- Multiple export formats (HTML, JSON, Markdown)
- Automated visualization generation
- Provenance tracking with cryptographic hashing
Healthcare AI/ML models require rigorous documentation for regulatory compliance, clinical validation, deployment safety, reproducibility, and transparency. This library automates the documentation process while maintaining flexibility for diverse use cases.
Installation
From PyPI
pip install smart-model-card
From Source
git clone https://github.com/ankurlohachab/smart-model-card.git
cd smart-model-card
pip install -e .
Optional Dependencies
OMOP integration support:
pip install smart-model-card[omop]
Visualization support:
pip install smart-model-card[viz]
Development tools:
pip install smart-model-card[dev]
All dependencies:
pip install smart-model-card[all]
Requirements
- Python >= 3.8
- requests >= 2.25.0
- smart-omop >= 0.1.0 (optional, for OMOP integration)
- matplotlib >= 3.3.0 (optional, for visualizations)
Quick Start
Interactive Wizard
Create a model card using the interactive CLI:
smart-model-card interactive
The wizard provides:
- Step-by-step prompts for all 7 sections
- Input validation (dates: YYYY-MM-DD, emails, numeric ranges)
- OMOP data integration option
- Default values for common fields
Example session:
SECTION 1: Model Details
Model Name: COPD-Risk-Predictor
Version [1.0.0]: 1.0.0
Release Date (YYYY-MM-DD): 2025-01-15
Support Contact (email): researcher@hospital.org
✓ Model Card Creation Complete!
Your model card has been saved to:
• HTML: /path/to/output/model_card.html
• JSON: /path/to/output/model_card.json
To view your model card:
open /path/to/output/model_card.html
Programmatic Usage
Basic example:
from smart_model_card import ModelCard, ModelDetails, IntendedUse
from smart_model_card.sections import DataFactors, SourceDataset
from smart_model_card.exporters import HTMLExporter
card = ModelCard()
card.set_model_details(ModelDetails(
model_name="Diabetes-Risk-Model",
version="2.1.0",
developer_organization="University Hospital Research Lab",
release_date="2025-01-15",
description="Predicts 5-year diabetes risk using EHR data",
intended_purpose="decision_support",
algorithms_used="XGBoost Classifier",
licensing="MIT",
support_contact="ai-team@hospital.org"
))
card.set_intended_use(IntendedUse(
primary_intended_users="Primary care physicians",
clinical_indications="Patients aged 40-75 with pre-diabetes indicators",
patient_target_group="Adults with BMI > 25 and family history of diabetes",
intended_use_environment="hospital_outpatient"
))
card.set_data_factors(DataFactors(
source_datasets=[
SourceDataset(
name="Hospital EHR Database",
origin="Academic Medical Center",
size=15000,
collection_period="2018-2023",
population_characteristics="Adult patients, 45% female, mean age 62"
)
],
data_distribution_summary="Balanced dataset with 30% positive cases",
data_representativeness="Representative of urban academic hospital population",
data_governance="IRB-approved, HIPAA-compliant data access"
))
HTMLExporter.export(card, "output/diabetes_model_card.html")
Features
Validation
The interactive wizard validates:
| Dates | YYYY-MM-DD format (e.g., 2025-01-15) |
| Emails | Standard email format (user@example.com) |
| Metrics | Numeric range 0.0-1.0 |
| Dataset Sizes | Integer >= 0 |
| Retry Limit | Maximum 5 attempts per field |
OMOP CDM Support
Integrate observational health data:
- Cohort extraction from OHDSI ATLAS
- Heracles report parsing for demographic characterizations
- Athena API integration for concept enrichment
- Automatic demographic table generation
- Age/gender/race distribution visualizations
Export Formats
HTML:
- Interactive collapsible sections
- Searchable tables
- Pagination support
- Embedded visualizations
- Multi-dataset dropdown switching
- Print-friendly styling
- Interactive tooltips for field descriptions
JSON:
- Structured schema
- Machine-readable format
- Version control friendly
- API-compatible
Markdown:
- Plain text format
- Git diff friendly
- Easy manual editing
Structure
All model cards include 7 sections:
- Model Details: Name, version, developer, release date, algorithms, licensing, contact
- Intended Use: Target users, clinical indications, patient population, environment, warnings
- Data & Factors: Source datasets, distribution, representativeness, governance
- Features & Outputs: Input features, output types, uncertainty quantification
- Performance & Validation: Validation datasets, metrics, calibration, fairness
- Methodology: Development workflow, training procedure, preprocessing, explainability
- Additional Information: Benefits, risks, ethics, limitations, recommendations
CLI Reference
Interactive Mode
smart-model-card interactive
Launch interactive wizard for model card creation.
Validation
smart-model-card validate model_card.json
Validate existing model card against schema.
Export
smart-model-card export model_card.json --format html -o output.html
smart-model-card export model_card.json --format json -o output.json
smart-model-card export model_card.json --format markdown -o output.md
Scaffold Generation
smart-model-card create --model-name "MyModel" -o scaffold.json
Generate template model card with placeholder values.
Provenance
smart-model-card hash --card model_card.json
smart-model-card diff old_card.json new_card.json
Fairness Analysis
smart-model-card fairness-check model_card.json
Analyze fairness metrics and demographic performance.
Python API
Core Classes
ModelCard
Main container for model card sections.
from smart_model_card import ModelCard
card = ModelCard()
card.set_model_details(details)
card.set_intended_use(use)
card.set_data_factors(data)
card.set_features_outputs(features)
card.set_performance_validation(perf)
card.set_methodology(method)
card.set_additional_info(info)
card_dict = card.to_dict()
ModelDetails
Section 1: Model information.
from smart_model_card import ModelDetails
details = ModelDetails(
model_name="MyModel",
version="1.0.0",
developer_organization="Org Name",
release_date="2025-01-15",
description="Model description",
intended_purpose="decision_support",
algorithms_used="Algorithm name",
licensing="MIT",
support_contact="email@org.com"
)
Purpose options: decision_support, screening, diagnosis, prognosis, other
IntendedUse
Section 2: Clinical context.
from smart_model_card import IntendedUse
use = IntendedUse(
primary_intended_users="Clinicians",
clinical_indications="Use cases",
patient_target_group="Patient population",
intended_use_environment="hospital_outpatient",
contraindications="When not to use",
out_of_scope_applications="Out of scope",
warnings="Important warnings"
)
Environment options: hospital_inpatient, hospital_outpatient, clinic, home, mobile, other
Exporters
from smart_model_card.exporters import HTMLExporter, JSONExporter, MarkdownExporter
HTMLExporter.export(card, "output/card.html")
JSONExporter.export(card, "output/card.json")
MarkdownExporter.export(card, "output/card.md")
OMOP Integration
Interactive Wizard
When creating a model card interactively, the wizard prompts for OMOP integration in Section 3:
Would you like to add OMOP data? [y/N]: y
Choose integration method:
1. Fetch existing cohort from OHDSI WebAPI
2. Create new cohort from scratch
3. Use locally saved cohort data
Select number: 1
OHDSI WebAPI URL: https://atlas.yourorg.org/WebAPI
Available CDM sources (2):
1. KAGGLECOPD - Kaggle COPD Dataset
2. SYNPUF1K - Synthetic Data
Select source number (1-2): 1
Cohort ID: 168
Include Heracles characterization reports? [Y/n]: y
✓ Successfully fetched cohort: COPD Patients 2023
✓ Cohort has 95 persons (status: COMPLETE)
The system includes:
- Automatic source discovery: Lists available CDM sources from WebAPI
- URL retry logic: Up to 3 attempts if connection fails
- Cohort creation: Build new cohorts interactively from scratch
- Person count validation: Verifies cohort generation status
Programmatic Integration
from smart_model_card.integrations import OMOPIntegration
with OMOPIntegration(
webapi_url="https://atlas.yourorg.org/WebAPI",
source_key="YOUR_CDM_SOURCE"
) as omop:
cohort_data = omop.get_cohort_with_reports(
cohort_id=168,
include_heracles=True
)
card.set_data_factors(cohort_data['data_factors'])
Available Reports
When Heracles characterization is included:
- Person: Age distribution, gender distribution, race/ethnicity
- Dashboard: Overview statistics
- Conditions: Diagnosis distributions
- Drugs: Medication exposures
- Procedures: Procedure utilization
Examples
The examples/ directory contains working demonstrations:
Basic Usage
python examples/quickstart.py
Creates minimal model card with required fields.
OMOP Integration
python examples/demo_smart_omop_integration.py
Demonstrates OMOP cohort fetching and demographic visualization generation.
Multi-Dataset
python examples/demo_multi_dataset.py
Documents models trained on multiple datasets with different characteristics.
Complete Workflow
python examples/demo_complete_workflow.py
End-to-end example including OMOP integration and multiple export formats.
Testing
Run Tests
pytest tests/ -v
Coverage Report
pytest tests/ --cov=smart_model_card --cov-report=html
Test Suite
- Model Card Validation (8 tests): Schema compliance, required fields
- Provenance Tracking (3 tests): Hash computation, version comparison
- CAC Integration (1 test): Code suggestion functionality
- Standardization (4 tests): Section structure, consistency checks
Total: 16 tests
Specific Tests
pytest tests/test_standardization.py -v
pytest tests/test_model_card.py -v
Development
Setup
git clone https://github.com/ankurlohachab/smart-model-card.git
cd smart-model-card
python -m venv venv
source venv/bin/activate
pip install -e .[dev]
pytest tests/
Code Style
This project follows PEP 8:
black src/ tests/
flake8 src/ tests/
Building
pip install build twine
python -m build
twine check dist/*
twine upload dist/*
Contributing
Contributions are welcome. Please:
- Fork the repository
- Create a feature branch (
git checkout -b feature/my-feature)
- Make changes
- Add tests for new functionality
- Ensure tests pass (
pytest tests/)
- Commit changes (
git commit -m 'Add my feature')
- Push to branch (
git push origin feature/my-feature)
- Open a Pull Request
See CONTRIBUTING.md for details.
License
MIT License - see LICENSE file.
Citation
If you use this library in research or projects, please cite:
@software{smart_model_card,
title={SMART Model Card: Standardized Model Documentation for Healthcare AI},
author={Lohachab, Ankur},
organization={Department of Advanced Computing Sciences, Maastricht University},
year={2025},
url={https://github.com/ankurlohachab/smart-model-card}
}
Support
Author
Ankur Lohachab,
Department of Advanced Computing Sciences,
Maastricht University