pandas-vet

pandas-vet

pandas-vet is a plugin for flake8 that provides opinionated linting for pandas code.

Basic usage

Take the following script, drop_column.py, which contains valid pandas code:

# drop_column.py
import pandas

df = pandas.DataFrame({
    'col_a': [i for i in range(20)],
    'col_b': [j for j in range(20, 40)]
})
df.drop(columns='col_b', inplace=True)

With pandas-vet installed, if we run Flake8 on this script, we will see three warnings raised.

$ flake8 drop_column.py

./drop_column.py:2:1: PD001 pandas should always be imported as 'import pandas as pd'
./drop_column.py:4:1: PD901 'df' is a bad variable name. Be kinder to your future self.
./drop_column.py:7:1: PD002 'inplace = True' should be avoided; it has inconsistent behavior

We can use these to improve the code.

# pandastic_drop_column.py
import pandas as pd

ab_dataset = pd.DataFrame({
    'col_a': [i for i in range(20)],
    'col_b': [j for j in range(20, 40)]
})
a_dataset = ab_dataset.drop(columns='col_b')

For a full list, see the Supported warnings page of the documentation.

Motivation

Starting with pandas can be daunting. The usual internet help sites are littered with different ways to do the same thing and some features that the pandas docs themselves discourage live on in the API. pandas-vet is (hopefully) a way to help make pandas a little more friendly for newcomers by taking some opinionated stances about pandas best practices. It is designed to help users reduce the pandas universe.

The idea to create a linter was sparked by Ania Kapuścińska's talk at PyCascades 2019, "Lint your code responsibly!". The package was largely developed at the PyCascades 2019 sprints.

Many of the opinions stem from Ted Petrou's excellent Minimally Sufficient Pandas. Other ideas are drawn from pandas docs or elsewhere. The Pandas in Black and White flashcards have a lot of the same opinions too.