The Open-source Framework for Precision Data Testing
📊 🔎 ✅
Data validation for scientists, engineers, and analysts seeking correctness.

pandera
is a Union.ai open
source project that provides a flexible and expressive API for performing data
validation on dataframe-like objects to make data processing pipelines more readable and robust.
Dataframes contain information that pandera
explicitly validates at runtime.
This is useful in production-critical or reproducible research settings. With
pandera
, you can:
- Define a schema once and use it to validate
different dataframe types
including pandas, polars,
dask, modin,
and pyspark.
- Check the types and
properties of columns in a
DataFrame
or values in a Series
. - Perform more complex statistical validation like
hypothesis testing.
- Parse data to standardize
the preprocessing steps needed to produce valid data.
- Seamlessly integrate with existing data analysis/processing pipelines
via function decorators.
- Define dataframe models with the
class-based API
with pydantic-style syntax and validate dataframes using the typing syntax.
- Synthesize data
from schema objects for property-based testing with pandas data structures.
- Lazily Validate
dataframes so that all validation checks are executed before raising an error.
- Integrate with
a rich ecosystem of python tools like pydantic,
fastapi, and mypy.
Documentation
The official documentation is hosted here: https://pandera.readthedocs.io
Install
Using pip:
pip install pandera
Using conda:
conda install -c conda-forge pandera
Installing additional functionality:
pip
pip install 'pandera[hypotheses]'
pip install 'pandera[io]'
pip install 'pandera[strategies]'
pip install 'pandera[mypy]'
pip install 'pandera[fastapi]'
pip install 'pandera[dask]'
pip install 'pandera[pyspark]'
pip install 'pandera[modin]'
pip install 'pandera[modin-ray]'
pip install 'pandera[modin-dask]'
pip install 'pandera[geopandas]'
pip install 'pandera[polars]'
conda
conda install -c conda-forge pandera-hypotheses
conda install -c conda-forge pandera-io
conda install -c conda-forge pandera-strategies
conda install -c conda-forge pandera-mypy
conda install -c conda-forge pandera-fastapi
conda install -c conda-forge pandera-dask
conda install -c conda-forge pandera-pyspark
conda install -c conda-forge pandera-modin
conda install -c conda-forge pandera-modin-ray
conda install -c conda-forge pandera-modin-dask
conda install -c conda-forge pandera-geopandas
conda install -c conda-forge pandera-polars
Quick Start
import pandas as pd
import pandera as pa
df = pd.DataFrame({
"column1": [1, 4, 0, 10, 9],
"column2": [-1.3, -1.4, -2.9, -10.1, -20.4],
"column3": ["value_1", "value_2", "value_3", "value_2", "value_1"]
})
schema = pa.DataFrameSchema({
"column1": pa.Column(int, checks=pa.Check.le(10)),
"column2": pa.Column(float, checks=pa.Check.lt(-1.2)),
"column3": pa.Column(str, checks=[
pa.Check.str_startswith("value_"),
pa.Check(lambda s: s.str.split("_", expand=True).shape[1] == 2)
]),
})
validated_df = schema(df)
print(validated_df)
DataFrame Model
pandera
also provides an alternative API for expressing schemas inspired
by dataclasses and
pydantic. The equivalent DataFrameModel
for the above DataFrameSchema
would be:
from pandera.typing import Series
class Schema(pa.DataFrameModel):
column1: int = pa.Field(le=10)
column2: float = pa.Field(lt=-1.2)
column3: str = pa.Field(str_startswith="value_")
@pa.check("column3")
def column_3_check(cls, series: Series[str]) -> Series[bool]:
"""Check that values have two elements after being split with '_'"""
return series.str.split("_", expand=True).shape[1] == 2
Schema.validate(df)
Development Installation
git clone https://github.com/pandera-dev/pandera.git
cd pandera
export PYTHON_VERSION=... # specify desired python version
pip install -r dev/requirements-${PYTHON_VERSION}.txt
pip install -e .
Tests
pip install pytest
pytest tests
Contributing to pandera 
All contributions, bug reports, bug fixes, documentation improvements,
enhancements and ideas are welcome.
A detailed overview on how to contribute can be found in the
contributing guide
on GitHub.
Issues
Go here to submit feature
requests or bugfixes.
Need Help?
There are many ways of getting help with your questions. You can ask a question
on Github Discussions
page or reach out to the maintainers and pandera community on
Discord
Why pandera
?
How to Cite
If you use pandera
in the context of academic or industry research, please
consider citing the paper and/or software package.
@InProceedings{ niels_bantilan-proc-scipy-2020,
author = { {N}iels {B}antilan },
title = { pandera: {S}tatistical {D}ata {V}alidation of {P}andas {D}ataframes },
booktitle = { {P}roceedings of the 19th {P}ython in {S}cience {C}onference },
pages = { 116 - 124 },
year = { 2020 },
editor = { {M}eghann {A}garwal and {C}hris {C}alloway and {D}illon {N}iederhut and {D}avid {S}hupe },
doi = { 10.25080/Majora-342d178e-010 }
}
Software Package

License and Credits
pandera
is licensed under the MIT license and is written and
maintained by Niels Bantilan (niels@union.ai)