The 3.0 update contains breaking changes:
poetry, mypy and black are no longer distributed with coveo-stew.
Please refer to the upgrade guide for more information and resolution steps.
coveo-stew delivers a complete Continuous Integration (CI) and Continuous Delivery (CD) solution using poetry as its backend.
Similar to: tox
pypi server)Similar to: poetry, flit, pbr, setuptools
Similar to: nothing! it's unique! 😎
Changed in 3.1: You need python 3.9+ to run stew, but you can still use it on projects requiring older versions of python.
Changed in 3.0: poetry is no longer provided out-of-the-box.
You need poetry installed on your system, and it must be available through the PATH.
The 3.0 version of coveo-stew is designed to work with modern versions of poetry.
While it is compatible with older versions, old poetry issues
such as this
and this
will cause stew to misbehave.
If you need to work with an older version of poetry,
consider using coveo-stew < 3.0 which had workarounds implemented around these issues.
Just like poetry, stew is a CLI tool that you install in your system.
It is recommended to install using pipx in order to isolate this into a nice little space:
pip install pipx --user
pipx install coveo-stew
If you don't use pipx, make sure to isolate the installation into a virtual environment.
This action checkouts the code, installs python, poetry and stew, and proceeds to run "stew ci" on a python project.
jobs:
stew-ci:
runs-on: ubuntu-latest
steps:
- uses: coveo/stew@main
with:
python-version: "3.10"
project-name: your-project-name
See additional options and documentation in the action file.
When running in a GitHub context, a step summary will automatically be generated.
This summary can be seen in the Summary tab of a workflow run view in GitHub.
You can disable this feature by launching stew ci with --no-github-step-report.
Please read these guides in order to learn how to organize your repository for maximum compatibility:
Unless a project name is specified, commands will operate on all projects in a git repository based on the current working folder:
stew <command>
stew <command> --help
Some commands allow specifying a project name:
stew <command> <project-name>
<project-name> in their name (partial, case-insensitive)stew ci tools will run on tools, tools-common, my-tools, etc.stew <command> <project-name> --exact-match
stew ci tools --exact-match will run on tools but not on tools-common, my-tools, etc.stew <command> .<path>
.stew ci . will only run on the project in the current folderstew ci ./tools will only run on the project in the tools folderThe main commands are explained below.
stew ciThe main show; it runs all CI tools on one or multiple projects.
Errors will show in the console, and junit xml reports will be generated inside the .ci folder.
Without configuration, this command will run the following checks:
mypy is not provided with coveo-stew. See builtin-runners for info.Options:
--fix will reformat the code if black fails. Additional fix routines may be added in the future.--check <runner> will launch only that runner. This option can be repeated.--skip <runner> will skip that runner. Takes precedence over --check. This option can be repeated.--quick skips running poetry install --remove-untracked before running the checks.
--quick is specified. See the quick configuration option.--no-github-step-report can be used to disable Step Report generation when running in a GitHub context.--extra <extra> (v3.1.1) will install the specified extra(s) for this run. Can be specified multiple times.--all-extras (v3.1.1) will install all extras for this run.--no-extras (v3.1.1) will not install any extra for this run.The configuration for this feature is explained in more details in the runners section.
stew buildStore the project and its locked dependencies to disk, so it can be installed without contacting a pypi server.
Optimally used to create truly repeatable builds and workflows (e.g.: containerized images, cloud storage, etc).
The folder can later be installed offline with pip install --no-index --find-links <folder> <project-name>.
Options:
--directory specifies where the wheels should be downloaded.--python may be used to target a different python. It's important to use the same python version, architecture and OS than the target system.Make sure your target <folder> is clean: Keep in mind that pip will still use the pyproject.toml constraints when installing, not poetry.lock.
The locked version system works when the locked version is the only thing that pip can find in the <folder>.
stew check-outdated and stew fix-outdatedChecks for out-of-date files or automatically update them.
Summary of actions:
poetry lock if pyproject.toml changed but not the poetry.lockstew pull-dev-requirements if a pydev project's dev-requirements are out of syncstew pull-dev-requirementsOnly useful on pydev projects (see about multiple-libraries).
It pulls the dev requirements from the local projects in order to aggregate them into the dev requirements of the root project.
Note: This command uses the dev group to identify test dependencies.
This can be defined as tool.poetry.dev-dependencies (poetry 1.1.15) or tool.poetry.group.dev.dependencies.
The group name cannot be customized, feel free to contribute the feature if that's useful to you.
stew bumpCalls poetry lock on all projects.
stew refreshCalls poetry install on all projects.
stew fresh-eggsClears the .egg-info folder from your projects. Usually followed by a poetry install or a stew refresh.
Use this if you change a [tool.poetry.scripts] section, else the changes will not be honored.
stew locate <project>Returns the path to a project:
$ stew locate coveo-stew
/home/jonapich/code/stew/coveo-stew
Configuration is done through the pyproject.toml file; default values are shown:
[tool.stew]
build-without-hashes = false
pydev = false
build-dependencies = {}
extras = []
all-extras = false
quick = {}
pip to download dependencies during stew build.stew build.
name = "version" or name = { version = "version", ... }stew build and stew ci.
stew ci --extra <this> --extra <that>.stew build and stew ci. Overrides the extras list.
stew ci --all-extras and stew ci --no-extras.stew ci --quick.
check or skip key, followed by a list of runners.--check and --skip options.quick = { skip = ["poetry-check", "check-outdated", "poetry-build", "pytest"] }quick = { check = ["mypy", "black"] }Configuration is done through each pyproject.toml file; default values are shown:
[tool.stew.ci]
mypy = true
pytest = false
black = false
poetry-check = true
check-outdated = true
offline-build = false
You don't have to include the [tool.stew.ci] section at all if these defaults suit you!
Changed in coveo-stew 3.0: mypy and black are no longer provided out-of-the-box.
In order to use a builtin or custom runner, you must have it installed. These locations are supported:
pyproject.toml)We strongly suggest pinning them to your
pyproject.toml file in the [tool.poetry.dev-dependencies] section.
This way, mypy won't surprise you with new failures when they release new versions! 😎
Note: You can override and customize most runners by rewriting them as custom runners.
A strict mypy configuration is provided.
You can provide your own mypy configuration, but you'll have to specify the set-config option:
[tool.stew.ci]
# disable stew's strict mypy config (i.e.: let mypy find its config)
mypy = { set-config = False }
# use a specific config (path relative to `pyproject.toml`'s folder)
mypy = { set-config = "mypy.ini" }
See https://mypy.readthedocs.io/en/stable/config_file.html#using-a-pyproject-toml
Pytest is configured to run with --doctest-modules --tb=short --durations=5 as well as JUnit report generation.
Some additional options are available:
[tool.stew.ci]
# configure the markers to test (i.e.: `-m`)
pytest = { marker-expression = 'not docker_tests' }
# disable the doctests
pytest = { doctest-modules = False }
Black supports the pyproject.toml file natively:
[tool.black]
line-length = 100
Ref: black documentation
Runs poetry check on each project.
Runs stew check-outdated.
Note: This runner cannot be overridden, but it can be disabled.
Runs stew build to a temporary folder and ensures that pip is able to reinstall everything from there.
Note: This runner cannot be overridden, but it can be disabled.
You can add your own runners to stew ci.
You can also redefine a builtin runner completely.
In this example, we create runners for flake8, bandit and isort. We also redefine the pytest runner:
[tool.stew.ci.custom-runners]
flake8 = true
bandit = { check-args = ["--quiet", "--recursive", "."] }
# some may prefer this toml syntax:
[tool.stew.ci.custom-runners.isort]
check-args = ["--check", ".", "--profile black"]
autofix-args = [".", "--profile black"]
[tool.stew.ci.custom-runners.pytest]
check-args = ["--tb=long", "--junitxml=.ci/pytest-results.xml"]
# using `executable`, you can create multiple custom runners with the same executable:
[tool.stew.ci.custom-runners.ruff-check]
executable = "ruff"
working-directory = "project"
check-args = ["check", "."]
autofix-args = [ "check", "--fix", "."]
[tool.stew.ci.custom-runners.ruff-format]
executable = "ruff"
working-directory = "project"
check-args = ["format", "--check", "."]
autofix-args = ["format", "."]
When a builtin runner such as pytest is redefined as a custom runner, you must provide all the arguments.
In this case, not passing --junitxml would mean that we lose the report that used to be in the .ci/ directory.
python -m <executable> first to see if it's installed in the virtual environment, else through the shell.executable, you can create multiple custom runners with the same executable (e.g.: ruff check vs ruff format)[1].pyproject.toml file. You can change it to "repository" in order to run from the root.executable.The args and check-args can be:
A string
A list of string:
When you call poetry install, you end up installing packages based on the poetry.lock file.
The resulting packages will always be the same, no matter what.
This is the dev scenario.
When you call pip install, you are installing packages based on the constraints placed in a pyproject.toml or a setup.py file.
Unless the constraints are hard pinned versions, the resulting packages are not guaranteed and will depend on the point in time when the installation is performed, among other factors.
This is the shared library scenario.
When you use poetry, you cover the two scenarios above.
The third scenario is the private business use case: you want to freeze your dependencies in time so that everything from the developer to the CI servers to the production system is identical.
Essentially, you want poetry install without the dev requirements.
This functionality is provided out of the box by stew build, which creates a pip-installable package from the lock file that you can then stash in a private storage of your choice or pass around your deployments.
You can keep poetry and stew off your production environment by creating a frozen archive of your application or library from your CI servers (docker used as example):
stew build tool which:
poetry build on your projectpip download based on the content of the lock file.wheels folder of your repo--python switch when calling stew build to specify which python executable to use! Make sure to use a python interpreter that matches the os/arch/bits of the system you want to provision.The content in .wheels can then be zipped and moved around. A typical scenario is to push it into a Docker Container:
.wheels folder into your Docker build context.wheels folderpip version! Either update it to latest, or pin it to something.python -m venv <location> to create a virtual environment natively.location/bin/python or location/Scripts/python.exe)<venv-python-executable> -m pip install <your-package> --no-index --find-links <wheels-folder-location>.wheels folder if you want. Consider keeping a copy of the lock file within the docker image, for reference.To make sure you use the python interpreter that matches the os/arch/bits of the system you want to provision, you can run stew build directly when building the container image.
In order to do so without packaging stew in production, you can use multi-stage builds.
If your IDE supports poetry, it should detect and use the pyproject.toml file.
To set it up manually:
poetry install from the location of the pyproject.toml filepoetry env list --full-path)Your IDE should proceed to analyze the environment and will pick up all imports automatically, regardless of your PYTHONPATH or your working directory. Since the local source is editable, any change to the source code will be reflected on the next interpreter run.
If you use the multiple-projects approach, you should hook your IDE to the pydev environment. See this documentation for more information.
Using the correct interpreter is all you need to do. There is no activation script or environment variables to set up: the interpreter executable inside the virtual environment folder is a fully bootstrapped and isolated environment.
<venv-python-exec> directly in the dockerfile's CMD<venv-python-exec>Use the -m switch in order to launch your app!
FAQs
Opinionated python packaging and development utilities
We found that coveo-stew demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.