A light weight tool for creating summary statistics from dataframes.
skimpy is a light weight tool that provides summary statistics about variables in pandas or Polars data frames within the console or your interactive Python window.
Think of it as a super-charged version of pandas' df.describe().
You can find the documentation here.
skim a pandas or polars dataframe and produce summary statistics within the console
using:
from skimpy import skim
skim(df)
where df is a pandas or polars dataframe.
If you need to a dataset to try skimpy out on, you can use the built-in test Pandas data frame:
from skimpy import generate_test_data, skim
df = generate_test_data()
skim(df)
╭──────────────────────────────────────────────── skimpy summary ─────────────────────────────────────────────────╮ │ Data Summary Data Types Categories │ │ ┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┏━━━━━━━━━━━━━┳━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━━┓ │ │ ┃ dataframe ┃ Values ┃ ┃ Column Type ┃ Count ┃ ┃ Categorical Variables ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩ ┡━━━━━━━━━━━━━╇━━━━━━━┩ ┡━━━━━━━━━━━━━━━━━━━━━━━┩ │ │ │ Number of rows │ 1000 │ │ float64 │ 3 │ │ class │ │ │ │ Number of columns │ 13 │ │ category │ 2 │ │ location │ │ │ └───────────────────┴────────┘ │ datetime64 │ 2 │ └───────────────────────┘ │ │ │ object │ 2 │ │ │ │ int64 │ 1 │ │ │ │ bool │ 1 │ │ │ │ string │ 1 │ │ │ │ timedelta64 │ 1 │ │ │ └─────────────┴───────┘ │ │ number │ │ ┏━━━━━━━━━━━━━━┳━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ mean ┃ sd ┃ p0 ┃ p25 ┃ p50 ┃ p75 ┃ p100 ┃ hist ┃ │ │ ┡━━━━━━━━━━━━━━╇━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━┩ │ │ │ length │ 0 │ 0 │ 0.5016 │ 0.3597 │ 1.573e-06 │ 0.134 │ 0.4976 │ 0.8602 │ 1 │ ▇▃▃▃▅▇ │ │ │ │ width │ 0 │ 0 │ 2.037 │ 1.929 │ 0.002057 │ 0.603 │ 1.468 │ 2.953 │ 13.91 │ ▇▃▁ │ │ │ │ depth │ 0 │ 0 │ 10.02 │ 3.208 │ 2 │ 8 │ 10 │ 12 │ 20 │ ▁▃▇▆▃▁ │ │ │ │ rnd │ 118 │ 11.8 │ -0.01977 │ 1.002 │ -2.809 │ -0.7355 │ -0.0007736 │ 0.6639 │ 3.717 │ ▁▅▇▅▁ │ │ │ └──────────────┴─────┴──────┴──────────┴────────┴───────────┴─────────┴────────────┴────────┴───────┴────────┘ │ │ category │ │ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ ordered ┃ unique ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩ │ │ │ class │ 0 │ 0 │ False │ 2 │ │ │ │ location │ 1 │ 0.1 │ False │ 5 │ │ │ └──────────────────────────────────┴───────────┴────────────────┴───────────────────────┴────────────────────┘ │ │ bool │ │ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ true ┃ true rate ┃ hist ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩ │ │ │ booly_col │ 516 │ 0.52 │ ▇ ▇ │ │ │ └────────────────────────────────────┴─────────────────┴───────────────────────────────┴─────────────────────┘ │ │ datetime │ │ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ first ┃ last ┃ frequency ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ │ │ datetime │ 0 │ 0 │ 2018-01-31 │ 2101-04-30 │ ME │ │ │ │ datetime_no_freq │ 3 │ 0.3 │ 1992-01-05 │ 2023-03-04 │ None │ │ │ └──────────────────────────────┴───────┴──────────┴────────────────────┴───────────────────┴─────────────────┘ │ │ <class 'datetime.date'> │ │ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ first ┃ last ┃ frequency ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩ │ │ │ datetime.date │ 0 │ 0 │ 2018-01-31 │ 2101-04-30 │ ME │ │ │ │ datetime.date_no_freq │ 0 │ 0 │ 1992-01-05 │ 2023-03-04 │ None │ │ │ └──────────────────────────────────┴───────┴──────────┴──────────────────┴──────────────────┴────────────────┘ │ │ timedelta64 │ │ ┏━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ mean ┃ median ┃ max ┃ │ │ ┡━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩ │ │ │ time diff │ 5 │ 0.5 │ 8 days 00:05:47 │ 0 days 00:00:00 │ 26 days 00:00:00 │ │ │ └──────────────────┴──────┴─────────┴───────────────────────┴───────────────────────┴────────────────────────┘ │ │ string │ │ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │ │ ┃ column_name ┃ NA ┃ NA % ┃ words per row ┃ total words ┃ │ │ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ │ │ text │ 6 │ 0.6 │ 5.8 │ 5761 │ │ │ └───────────────────────────┴─────────┴────────────┴──────────────────────────────┴──────────────────────────┘ │ ╰────────────────────────────────────────────────────── End ──────────────────────────────────────────────────────╯
It is recommended that you set your datatypes before using skimpy (for example converting any text columns to pandas string datatype), as this will produce richer statistical summaries. However, the skim() function will try and guess what the datatypes of your columns are.
You can find a full list of requirements in the pyproject.toml file.
You can try this package out right now in your browser using this Google Colab notebook (requires a Google account). Note that the Google Colab notebook uses the latest package released on PyPI (rather than the development release).
You can install the latest release of skimpy via pip from PyPI:
$ pip install skimpy
To install the development version from git, use:
$ pip install git+https://github.com/aeturrell/skimpy.git
For development, see contributing.
Distributed under the terms of the MIT license, skimpy is free and open source software.
If you encounter any problems, please file an issue along with a detailed description.
This project was generated from @cjolowicz's Hypermodern Python Cookiecutter template.
skimpy was inspired by the R package skimr and by exploratory Python packages including ydata_profiling and dataprep, from which the clean_columns function comes.
This package would not have been possible without the Rich package.
The package is built with poetry, while the documentation is built with Quarto and Quartodoc (a Python package). Tests are run with nox.
Using skimpy in your paper? Let us know by raising an issue beginning with "citation" and we'll add it to this page.
FAQs
skimpy
We found that skimpy demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.