Djblets has historically been a setuptools-based project, relying

heavily on the dynamic ability of setup.py. Over the years, the Python

ecosystem has moved to pyproject.toml files, with pluggable build

backends. This has reached a point of maturity, and pip will soon

remove support for installing either production or editable installs

from a legacy source tree.
In theory, modernization requires just providing a pyproject.toml that

specifies setuptools as a build backend. A project can still use

setup.py for the project definition and dynamic capabilities. However,

since packages are also now built in a virtualenv, it's become clear

that we needed to address about our packaging.
We now use pyproject.toml to define most of the metadata of the

package. The build backend is then a specialization of

setuptools.build_meta, living in build-backend.py at the root of the

tree. This specializes a few things about our build process:


It uses all of Djblets's dependencies as build-system dependencies,

   needed in order to build static media for the package.


It builds the static media, including them in both sdist and wheel

   distributions.


It writes out a package-requirements.txt at build time with the

   dependencies from djblets/dependencies.py, which setuptools can

   then consume. This is also bundled in the sdist.


With this, we no longer need setup.py, and constrain all custom logic

to build-backend.py.
Some things to note:


python -m build . will default to building an sdist and then a

   wheel from that sdist, whereas build . -w will build a wheel

   straight from the tree. Due to differences in the prep stages, and

   what's built from what, we need to micromanage some state (like

package-requirements.txt) in different places.


Other build backends (hatch, PDM, flit, and poetry) were evaluated

   and discarded. These are great backends, but don't solve the core

   problems we've had to work around out of the box, and sort of want to

   manage more of the development and build process. setuptools is a

   known entity for us, and will be needed for extension packaging

   anyway, so we're sticking with it.


Makefile installs the package in editable-compat mode. This uses

   standard .pth files instead of newer setuptools-based import

   hooks, in order to allow mypy and pyright to find type

   definitions. See https://github.com/python/mypy/issues/13392 and

   https://github.com/pypa/setuptools/issues/3518.

Tested isolated builds in the following setups:

python -m build . (builds sdist, then a wheel from the sdist)
python -m build . -s (builds sdist)
python -m build . -w (builds wheel)

Tested isolated editable installs using pip install -e . and

non-isolated editable installs using pip install -e --no-build-isolation

and with make develop.
Performed full tree diffs of generated wheels from ./setup.py bdist_wheel

(prior to this change) and both wheel-producing python -m build methods.

Verified that all content was identical, with the exception of differences

in source map paths and some modern metadata for the package.