Rework `setup.py develop` for build isolation and local Djblets support.

Review Request #14065 — Created July 31, 2024 and submitted — Latest diff uploaded

Information

Review Board
release-7.x

Reviewers

Review Board's packaging requires a local build of Djblets, as it makes
use of metadata and packaging functionality that Djblets provides. We
often need to test Djblets changes with Review Board, so being able to
install against a local editable Djblets package is important, and for
this reason, we turned off build isolation.

Unfortunately, this in turn broke Haystack installation. Haystack
doesn't provide wheels, just a tarball, and this tarball uses
setuptools-scm as part of the build process. What setuptools-scm
does is look for a local tag or branch that indicates the package
version, and falls back to a default of 0.0.0 if not found.

When not run in build isolation mode, the packaging in the tarball ends
up looking at Review Board's tree, computes 0.0.0 as the version, and
this fails to satisfy the version range pip is considering, which then
fails installation.

This means we need build isolation, but we also need local Djblets.

We're now trying a new approach. We now generate a temporary
requirements.txt that includes:

  1. The Djblets source tree as an editable (if found as editable already)
  2. dev-requirements.txt
  3. doc-requirements.txt, if needed

We then pass that to pip, in the default build isolation mode. This
allows pip to consider all the dependencies across all these at once.
It also ensures that a local editable of Djblets will be used as
editable. It does still need to install its dependencies in the build
isolation environment, but as this generally uses cached wheels, it's
pretty fast.

The result should be the same as before, but without the side effects
Haystack suffers from and without pulling down a published Djblets.

Created new virtual environments and installed in the following
conditions:

  1. With a local Djblets in editable mode.
  2. With a parent environment install of published Djblets.
  3. Without any existing install of Djblets.

Verified that in the first case, the existing Djblets tree is
used as part of package building with minimal setup time, and
that in the other cases a published Djblets is used instead.

Commits

Files