setuptools-github

setuptools-github

Introduction

setuptools-github helps to setup a simple project life cycle where the target is delivering packages into PyPI from a hosted project at Github.

The idea is rather simple (and detailed in here):

• commits on a master branch will trigger code checks (static checks, tests etc.)
• commits on a beta/N.M.O branch will do all the previous checks + publishing a beta package N.M.ObXXX (XXX is an increasing number) on PyPI
• tagging on a beta/N.M.O branch will publish an official package on PyPI for N.M.O

See here for what the life cycle implementation looks like.

Index

1. Setup the project
   • install the package
   • setup the init.py file
   • fix the setup.py file
2. Setup the workflow files
   • add the files
   • setup the secrets
3. Working with branches
   • commit on the master branch
   • commit on a beta/N.M.O branch
   • releasing on tags

Setup the project

The project should conform to this layout style:

  project-name/
  ├── setup.py
  ├── pyproject.toml
  ├── .github
  │   └── workflows           <- workflow files for
  │       ├── beta.yml             * beta/N.M.O branches
  │       ├── master.yml           * master branch
  │       └── tags.yml             * release/N.M.O tags
  ├── src
  │   └── project_name        <- project name
  │       └── __init__.py     <- version_file
  └── tests                   <- tests (pytest)
      ├── conftest.py
      └── requirements.txt    <- requirement file for tests

NOTE for a pyproject.toml / hatch enabled version of this, please use hatch-ci plugin

install the package

pip install setuptools-github
 or
conda install -c conda-forge setuptools-github

setup up the initial version_file

Create a new version_file src/project_name/__init__.py file to store the package information:

__version__ = "N.M.O"  # replace N, M and O with numerical values (eg. 0.0.0)
__hash__ = ""  # leave this empty

Fix the setup.py file

Include in the setup.py file:

from setuptools_github import tools

setup(
  name="project-name",
  version=tools.process(version_file, os.getenv("GITHUB_DUMP"))["version"],
  ...

NOTE: there's an annotated tools.process example in setup.py with support for keyword substitution on text files.

Setup the workflow files

These are the steps to automate the build process on github.

add workflow files

Add these workflows file to your project:

• github/workflows/master.yml
• github/workflows/beta.yml
• github/workflows/tags.yml

These will trigger a build on:

• a master branch commit see
• a beta/N.M.O commit see
• a release on tag release/N.M.O see

NOTE: Most likely you might need to change:

• the tests/requirements.txt file
• the envs variables at the beging of master.yml and beta.yml

Setup github secrets

In order to publish to codecov the coveragen info and to PyPI the wheels, you need to set the github secrets under:

https://github.com/username/project-name/settings/secrets/actions

These are the needed secrets for the PyPI index and codecov services:

• TWINE_PASSWORD
• TWINE_USERNAME
• CODECOV_TOKEN

---

THAT IS ALL! Now when commit to the master branch, this will trigger the github action to run tests and quality checks on the code

Working with branches

commit on the master branch

Every time there's a commit on the master branch, this will trigger the workflow under ./github/workflows/master.yml:

• Runs mypy on src/
• Runs ruff on src/
• Run all tests under tests/

On completion static and dynamic tests are supported.

commit on a beta/N.M.O branch

In order to prepare for a release a new beta/N.M.O branch should be created:

python -m setuptools_github.script make-beta src/project_name/__init__.py 
or
setuptools-github make-beta src/project_name/__init__.py

Every commit on beta/N.M.O branch if Secrets have been set properly:

• Runs mypy on src/
• Runs ruff on src/
• Run all tests under tests/
• Run coverage on tests/
• Send the coverage result into coverage
• Create a new wheel package under dist/
• (on success) Send the new wheels package-N.M.O.bX to PyPI

NOTE: the name project-N.M.O.bX contains the X: this is an incrementing counter set during build. This means project-N.M.O.bX < project-N.M.O allowing the correct package ordering.

releasing on tags

To release an official package for project-N.M.O from the beta/N.M.O branch:

python -m setuptools_github.script micro src/project_name/__init__.py
or
setuptools-github make-beta micro src/project_name/__init__.py

This will tag the HEAD on beta/N.M.O branch with the release/N.M.O tag and increment the version_file with the next version N.M.O+1 (using micro).

Once done, you'll need to push it the tag.

git push release/N.M.O

This will:

• trigger a CI build that will create the project-name-N.M.O
• Create a new wheel package under dist/
• (on success) Send the new wheels project-N.M.O to PyPI