zc.buildout

Buildout

.. image:: https://github.com/buildout/buildout/actions/workflows/run-tests.yml/badge.svg :alt: GHA tests report :target: https://github.com/buildout/buildout/actions/workflows/run-tests.yml

Buildout is a project designed to solve 2 problems:

• Application-centric assembly and deployment

  Assembly runs the gamut from stitching together libraries to create a running program, to production deployment configuration of applications, and associated systems and tools (e.g. run-control scripts, cron jobs, logs, service registration, etc.).

  Buildout might be confused with build tools like make or ant, but it is a little higher level and might invoke systems like make or ant to get its work done.

  Buildout might be confused with systems like puppet or chef, but it is more application focused. Systems like puppet or chef might use buildout to get their work done.

  Buildout is also somewhat Python-centric, even though it can be used to assemble and deploy non-python applications. It has some special features for assembling Python programs. It's scripted with Python, unlike, say puppet or chef, which are scripted with Ruby.

• Repeatable assembly of programs from Python software distributions

  Buildout puts great effort toward making program assembly a highly repeatable process, whether in a very open-ended development mode, where dependency versions aren't locked down, or in a deployment environment where dependency versions are fully specified. You should be able to check buildout into a VCS and later check it out. Two checkouts built at the same time in the same environment should always give the same result, regardless of their history. Among other things, after a buildout, all dependencies should be at the most recent version consistent with any version specifications expressed in the buildout.

  Buildout supports applications consisting of multiple programs, with different programs in an application free to use different versions of Python distributions. This is in contrast with a Python installation (real or virtual), where, for any given distribution, there can only be one installed.

To learn more about buildout, including how to use it, see https://www.buildout.org/.

Native namespaces and breaking changes in 5.x

Summary: zc.buildout 5.x installs most distributions with pip, even editable installs (with some remarks). It automatically uses native namespaces for all packages, except editable installs. Eggs go to directory eggs/v5 to avoid compatibility problems.

If a package name has a dot in it, it is a namespace package. zc.buildout has zc as namespace. Namespaces can be implemented in three ways: native namespaces (PEP 420, since Python 3.3), pkg_resources style (with __init__.py files that call pkg_resources.declare_namespace) and pkgutil style (with __init__.py files that call pkgutil.extend_path).

Native namespaces are the modern way to do it. They work better with pip and other modern tools. The pkg_resources style depends on setuptools and is considered deprecated: setuptools is scheduled to drop support for it at the end of 2025, removing its foundational pkg_resources module.

Problems start when you have multiple packages in the same namespace, that use different implementations. But it depends on what you use to install the packages. In the following examples, we have two packages in the same namespace, say ns.native (using native namespaces) and ns.deprecated (using pkg_resources style).

• Make editable installs of both packages (pip install -e or in buildout, develop =):

  • This works neither in pip nor in buildout.
  • You can install the horse-with-no-namespace <https://pypi.org/project/horse-with-no-namespace/>_ package to get this working.

• Make a normal install of both packages:

  • This works fine in pip.
  • This fails in buildout 4.x.
  • This works fine in buildout 5.x.

• Make a normal install of one package and an editable install of the other:

  • This works fine in pip.
  • This fails in buildout 4.x.
  • This fails in buildout 5.x as well. But again, you can use horse-with-no-namespace to get this working.

So the big difference between buildout 4.x and 5.x is that with 5.x you can mix namespace styles when you do normal installs. This is possible due to the following major changes in 5.x:

• The zc.buildout package itself uses native namespaces now.
• zc.buildout 5.x installs most packages with pip, with only rare exceptions.
• The tests have mostly been changed to use wheels instead of eggs, so they more closely resemble real life.
• Buildout now automatically treats all namespace packages as having native namespaces. Previously, after we installed a pkg_resources-style namespace package with pip, the __init__.py files would be missing, so we would explicitly add them again. Now we no longer do this. In fact, in some cases these files are there after installation, and we explicitly remove them.
• This means that "eggs" created by this Buildout version are not compatible with eggs from a previous Buildout version, at least for namespace packages. So Buildout now stores the eggs in a sub directory: eggs/v5. Or with abi tags for example: eggs/v5/cp313. So it should be fine to keep using the same shared eggs cache, even if you are using different zc.buildout versions.
• Development eggs (editable installs) are now also installed by pip. Previously, zc.buildout would call python setup.py develop to install them, failing if there was no setup.py file. Now this file is no longer needed. Theoretically this means that you could develop a package that uses for example hatchling as build system. In practice this does not work yet, but the foundation is there.

The Plone project is a major user of buildout, although alternative installation methods are becoming more popular. So what does this mean for Plone?

• Plone 6.1 and earlier will keep using zc.buildout 4.x and pkg_resources-style namespaces
• Plone 6.2 will switch to zc.buildout 5.x.
• Plone 6.2 will temporarily use horse-with-no-namespace both when installed with pip and with zc.buildout.
• This combination means that each package in Plone 6.2 can switch to native namespaces at its own pace. Previously, all packages in a namespace had to switch at the same time, and the tests would be broken until that happened.
• In your own projects on any Plone 6 version, you are free to use any buildout and setuptools version you want. If Buildout 5 works on your Plone 6.0 project, or Buildout 4 works on your Plone 6.2 project, that is fine.

Change History

.. You should NOT be adding new change log entries to this file. You should create a file in the news directory instead. For helpful instructions, please see: https://github.com/buildout/buildout/blob/master/doc/ADD-A-NEWS-ITEM.rst

.. towncrier release notes start

5.2.0 (2026-04-29)

New features:

• Restrict setuptools to less than 82 if it is not pinned. When zc.buildout checks if it should upgrade itself or setuptools, under some circumstances this could lead to a too new setuptools version getting installed. [maurits]

Tests:

• Test with pip 26.1. [maurits]

5.1.3 (2026-03-06)

Bug fixes:

• Replace 4 bare except clauses with except Exception. [haosenwang1018]

5.1.2 (2026-02-10)

Bug fixes:

• Accept setuptools 81. Require setuptools<82. [maurits]

5.1.1 (2025-11-25)

Bug fixes:

• Store buildout index url in the installer, so pip can use it. [maurits] (#731)
• When installing packages in development, treat the name as a file uri. Otherwise pip install -e package_name will look on PyPI, instead of a local package_name directory. [maurits] (#734)

5.1.0 (2025-11-20)

New features:

• Support Python 3.14. No changes were needed. [maurits] (#314)
• Warn when old-style namespaces are used in packages under development. [maurits] (#729)

Bug fixes:

• Require setuptools<81. We need the pkg_resources module which is scheduled for removal in 81. [maurits] (#81)

5.0.0 (2025-11-12)

Tests:

• Test with pip 25.3. (#727)

5.0.0a3 (2025-09-24)

Bug fixes:

• Fix logic in detecting namespace init files for deletion. [maurits] (#720)

5.0.0a2 (2025-09-11)

Bug fixes:

• Fix reading metadata files from dist-info in Windows. [maurits] (#722)

5.0.0a1 (2025-09-11)

Breaking changes:

• Install development eggs (editable installs) using pip. Theoretically this means that you could develop a package that uses for example hatchling as build system. In practice this does not work yet, but the foundation is there. [maurits] (#676)
• Install all namespace packages as native namespaces. [maurits] (#676)
• Store eggs in a sub directory: eggs/v5. Or with abi tags for example: eggs/v5/cp313. [maurits] (#676)
• Install most packages with pip, with only rare exceptions. [maurits] (#676)
• The zc.buildout package itself uses native namespaces now. [maurits] (#676)
• Require at least setuptools version 61.0.0. This is needed due to the changes in the test setup. [maurits]

Tests:

• The tests have mostly been changed to use wheels instead of eggs, so they more closely resemble real life. [maurits] (#676)
• Removed inactive tests for no longer existing bootstrap.py. [maurits]
• Split the buildout.txt test file into multiple files. [maurits]

4.1.12 (2025-06-11)

Bug fixes:

• Fix error get_win_launcher not found on Windows on setuptools 80.3+. [maurits] (#713)

4.1.11 (2025-06-11)

Bug fixes:

• Fix development installs to still work when using setuptools 80.0.0. From then on, setuptools internally calls pip install --editable. Note that "distutils scripts" can no longer be detected with setuptools 80. This seems an ancient technology, and probably hardly used. [maurits] (#708)
• Use a copy of package_index.py from setuptools 80.2.0. This fixes compatibility with setuptools 80.3.0 where this module was removed. Merged some of our patches into this copy. [maurits] (#710)

4.1.10 (2025-05-21)

Bug fixes:

• Override pkg_resources.Environment.can_add to have better results on Mac. Without this, a freshly created Mac-specific egg may not be considered compatible. This can happen when the Python you use was built on a different Mac OSX version. [maurits] (#609)

4.1.9 (2025-04-09)

Bug fixes:

• Fix accidental changes to PYTHONPATH in os.environ when calling pip install. [xavth] (#639)

Tests

• Use wheel 0.45.1 when testing with setuptools older than 70.1.0. Otherwise, when combining an older setuptools with a newer wheel version, the bdist_wheel command exists in neither of these packages. [maurits] (#705)

4.1.8 (2025-04-09)

Bug fixes:

• Use the canonical name of a package when checking for a version constraint. [maurits] (#689)
• Get actual project name from dist. Use this for naming the egg that gets created after installing a wheel or after doing a pip install of a source dist. [maurits] (#695)
• Log all http errors when processing package url. [maurits] (#1013)

4.1.7 (2025-04-08)

Bug fixes:

• Prevent getting package pages twice. Since version 4.1.5 we first request normalized package url on PyPI servers, but a subsequent check needed a fix. [maurits] (#634)
• No longer recompile py files if we moved the dist. This code was never updated for Python 3, where the .pyc files are in a __pycache__ directory, so it had no effect. [maurits] (#699)
• Require at least packaging version 23.2. Needed because we use the utils.is_normalized_name function. [maurits] (#700)

4.1.6 (2025-04-03)

Tests

• While creating sample packages for testing, mostly create wheels instead of eggs. For the sample source distributions, create tar.gz instead of zip files. Then our package index for testing is more like the actual PyPI. [maurits] (#675)

4.1.5 (2025-03-31)

Bug fixes:

• Implement PEP 503: request normalized package url on PyPI servers. [andreclimaco] (#634)
• Install wheel before setuptools when checking if an upgrade and restart are needed. [maurits] (#691)

4.1.4 (2025-03-07)

Bug fixes:

• If needed, copy and rename wheels before making an egg out of them. This helps for wheels of namespace packages created with setuptools 75.8.1 or higher. For namespace package we need a dot instead of an underscore in the resulting egg name. [maurits] (#686)

4.1.3 (2025-03-05)

Bug fixes:

• Patch the find method from pkg_resources.WorkingSet. Let this use the code from setuptools 75.8.2, if the currently used version is older. This is better at finding installed distributions. But don't patch setuptools versions older than 61: the new version of the method would give an error there. [maurits] (#682)

4.1.2 (2025-03-05)

Bug fixes:

• Fix error finding the zc.buildout distribution when checking if we need to upgrade/restart. This depends on your setuptools version. [maurits] (#681)

4.1.1 (2025-03-04)

Bug fixes:

• Fix error adding minimum zc.buildout version as requirement. [maurits] (#679)

4.1 (2025-03-04)

New features:

• In the ls testing method, add keyword argument lowercase_and_sort_output. The default is False, so no change. When true, as the name says, it sorts the output by lowercase, and prints it lowercase. We need this in one test because with setuptools 75.8.1 we no longer have a filename MIXEDCASE-0.5-pyN.N.egg, but mixedcase-0.5-pyN.N.egg. [maurits] (#7581)

Bug fixes:

• When trying to find a distribution for package.name, first try the normalized name (package_name). This fixes an error finding entry points for namespace packages. The error is: TypeError: ('Expected str, Requirement, or Distribution', None). [maurits] (#7581)

Development:

• Test with latest setuptools 75.8.2 and with pip 25.0.1. Note that setuptools 75.8.1 can be troublesome and should be avoided. [maurits] (#7581)

4.0 (2025-01-30)

Breaking changes:

• Drop Python 3.8 support. Require 3.9 as minimum. (#38)

Development:

• Test against setuptools == 75.6.0. (#671)

4.0.0a1 (2024-10-22)

Breaking changes:

• Add dependency on packaging. This gets rid of ugly compatibility code. [maurits] (#38)
• Require setuptools >= 49.0.0. This is the first version that supports PEP 496 environment markers, for example demo ==0.1; python_version < '3.9'. An earlier change had setuptools >= 42.0.2, otherwise we got ImportErrors. Also, since this is higher than 38.2.3, we are sure to have support for wheels. Remove support for distribute, which was probably already broken. [maurits] (#38)
• Drop support for Python 2. Require Python 3.8 as minimum. [maurits] (#38)

New features:

• Support Python 3.12 and 3.13. This only needed a few test fixes. [maurits] (#38)