Switch Review Board to a pyproject.toml-based packaging setup.

Review Request #14291 — Created Jan. 14, 2025 and updated — Latest diff uploaded

Information

Review Board
release-7.1.x

Reviewers

With legacy packaging centered around setup.py now on the way out, we
need to move to modern packaging using pyproject.toml and build
backends. The challenge is that Review Board's build setup is a bit more
involved than your average Python package, with static media building
and gettext i18n file generation being part of the process, and a fair
amount of setup required to get there.

This change introduces a build backend based on the work we did for
Djblets's backend, but with a few extra capabilities to ease
development and support the setup requirements, and doing so in pip's
isolated build environment.

The build backend handles a few custom steps:

  1. Dependency management
  2. Static media setup and building
  3. i18n building

Dependency management is a bit special, supporting testing against local
development packages.

When building a package, we pull dependencies from
reviewboard/dependencies.py and serialize them to
package-requirements.txt. This file is then consumed by the underlying
setuptools backend, turning into the list of dependencies for a package.

When installing in editable mode, it goes through the list of
dependencies and looks for any that has a matching symlink in
.local-packages/.

If one is found, then that path is used for the dependency, allowing us
to build and test against a local package. We do this automatically for
Djblets when invoked by ./contrib/internal/prepare-dev.py.

The result is then used as the list of dependencies for the temporary
build environment, and for the .npm-workspaces symlinks.

Static media setup uses .npm-workspaces as before, but it's now built
automatically when we pip install -e or build a package.

Note that any symlinks in .npm-workspaces will get replaced with links
to temporary locations when building a package, but this is all the more
reason to only build a package in a clean clone (beanbag-tools does
this automatically).

Along with these changes, MANIFEST.in has been updated with several
files and exclusions needed to generate a suitable source distribution.
We previously had files included that should not have been there and
other files missing. The list is now more comprehensive.

Given the straightjacket that is the Python build environment, some of
these specialized steps are very non-standard. Though this should not
break, it does require further testing. We'll aim to ship this as part
of Review Board 7.1.

Tested setting up a new development tree using prepare-dev.py with a
local Djblets tree (and a public build, but we can't build static media
with that, lacking shipped build tools in the Wheel). Verified that I
could run Review Board and all static media loaded, and that the links
in .local-packages and .npm-workspaces were correct.

Tested manually setting up editable mode without prepare-dev.py.

Tested building wheels and source distributions (together and
independently), and compared to the setup.py-based builds. This was
tested in both a clean Git clone and my development tree. I checked
the files in the builds for expected content.

Diff Revision 1

This is not the most recent revision of the diff. The latest diff is revision 2. See what's changed.

orig
1
2

Commits

First Last Summary ID Author
Switch Review Board to a pyproject.toml-based packaging setup.
With legacy packaging centered around `setup.py` now on the way out, we need to move to modern packaging using `pyproject.toml` and build backends. The challenge is that Review Board's build setup is a bit more involved than your average Python package, with static media building and gettext i18n file generation being part of the process, and a fair amount of setup required to get there. This change introduces a build backend based on the work we did for Djblets's backend, but with a few extra capabilities to ease development and support the setup requirements, and doing so in pip's isolated build environment. The build backend handles a few custom steps: 1. Dependency management 2. Static media setup and building 3. i18n building Dependency management is a bit special, supporting testing against local development packages. When building a package, we pull dependencies from `reviewboard/dependencies.py` and serialize them to `package-requirements.txt`. This file is then consumed by the underlying setuptools backend, turning into the list of dependencies for a package. When installing in editable mode, it goes through the list of dependencies and looks for any that has a matching symlink in `.local-packages/`. If one is found, then that path is used for the dependency, allowing us to build and test against a local package. We do this automatically for Djblets when invoked by `./contrib/internal/prepare-dev.py`. The result is then used as the list of dependencies for the temporary build environment, and for the `.npm-workspaces` symlinks. Static media setup uses `.npm-workspaces` as before, but it's now built automatically when we `pip install -e` or build a package. Note that any symlinks in `.npm-workspaces` will get replaced with links to temporary locations when building a package, but this is all the more reason to only build a package in a clean clone (`beanbag-tools` does this automatically). Along with these changes, `MANIFEST.in` has been updated with several files and exclusions needed to generate a suitable source distribution. We previously had files included that should not have been there and other files missing. The list is now more comprehensive. Given the straightjacket that is the Python build environment, some of these specialized steps are very non-standard. Though this should not break, it does require further testing. We'll aim to ship this as part of Review Board 7.1.
a30808bb0969f4c9bb613756f8d979d511e5103f Christian Hammond
.gitignore
MANIFEST.in
build-backend.py
pyproject.toml
setup.cfg
setup.py
contrib/internal/build-i18n.py
contrib/internal/build-media.py
contrib/internal/prepare-dev.py
reviewboard/dependencies.py
Loading...