• 
      

    Add the buildthings build backend.

    Review Request #14951 — Created March 20, 2026 and submitted

    Information

    buildthings
    master

    Reviewers

    Buildthings is a setuptools-based Python build backend that makes it
    easier to work with projects with more complex build requirements,
    particularly those involving NPM or packages that may still be in
    development in local source trees.

    It's based on the build backends we introduced in Review Board and
    Djblets, but more refined and capable of being expanded for additional
    use cases down the road.

    This initial version supports:

    • .local-packages/, mapping dependency names to local source trees,
      used in isolated build environments (available per-build type).

    • Options for including dev- and install-time dependencies within the
      isolated build environments (available per-build type).

    • An option for excluding certain, more complex dependencies from the
      isolated environments (available per-build type).

    • Options for enabling NPM installs and for managing workspaces based on
      other Python packages.

    • Support for extra build steps for editables, wheels, and source
      distributions.

    Much of this behaves the way it did in our in-tree build backends, but
    with several behavior issues fixed. For example, development
    dependencies will now map to local paths if found in .local-packages/.
    Further, depending on an editable package using this backend will no
    longer error out when it in turn depends on other local trees.

    Full documentation for all the options are available in the README.

    Converted Djblets and Review Board over to buildthings.

    Tested building in new virtualenvs using editable mode, ensuring
    that .local-packages was picked up, and using the install deps
    options.

    Checked that when building Review Board, the Djblets .local-packages
    was used.

    Verified that the local packages weren't installed as package
    dependencies (which would override any editable installs).

    Tested various combinations of the options and overrides.

    Tested building sdist and wheel distributions, with and without
    .local-packages and other deps turned on.

    Tested the build steps support.

    Tested that when building packages with include-{dev,install}-deps,
    those packages were installed in the environment but that the
    development deps didn't become part of the final package's dependency
    list.

    Summary ID
    Add the buildthings build backend.
    Buildthings is a setuptools-based Python build backend that makes it easier to work with projects with more complex build requirements, particularly those involving NPM or packages that may still be in development in local source trees. It's based on the build backends we introduced in Review Board and Djblets, but more refined and capable of being expanded for additional use cases down the road. This initial version supports: * `.local-packages/`, mapping dependency names to local source trees, used in isolated build environments (available per-build type). * Options for including dev- and install-time dependencies within the isolated build environments (available per-build type). * An option for excluding certain, more complex dependencies from the isolated environments (available per-build type). * Options for enabling NPM installs and for managing workspaces based on other Python packages. * Support for extra build steps for editables, wheels, and source distributions. Much of this behaves the way it did in our in-tree build backends, but with several behavior issues fixed. For example, development dependencies will now map to local paths if found in `.local-packages/`. Further, depending on an editable package using this backend will no longer error out when it in turn depends on other local trees. Full documentation for all the options are available in the README.
    507d56edb28f26676642250128dd531a02ad1abb
    Description From Last Updated

    We need to add toml as a dependency for python_version<3.11

    daviddavid

    There's no override of prepare_metadata_for_build_sdist. Should we have one?

    daviddavid

    This should probably have a line of ===== above it too.

    daviddavid

    'from setuptools.build_meta import *' used; unable to detect undefined names Column: 1 Error code: F403

    reviewbotreviewbot

    'setuptools.build_meta.*' imported but unused Column: 1 Error code: F401

    reviewbotreviewbot

    Looks like some of this might have been copy/pasted from sdist

    daviddavid

    Are these things actually necessary for the metadata operation? This is pretty heavyweight, and seems like it could be limited …

    daviddavid

    The local_packages_path in isolation_config is str | None but apply_local_dep_paths types this as str

    daviddavid

    This will end up computing npm_workspaces_dir relative to buildthings/backend.py. I think we want the cwd instead.

    daviddavid

    Because command is an str, this needs shell=True

    daviddavid

    We should pass text=True to subprocess.run(), otherwise the error will look like this: Failed to install npm packages: b'...'

    daviddavid

    Shouldn't this be ImportError?

    daviddavid

    This cache works when we're doing a single type of build, but in the case where a package has different …

    daviddavid

    This drops invalid requirements lines silently.

    daviddavid

    This should be able to just be < instead of <=

    daviddavid

    'from setuptools.build_meta import *' used; unable to detect undefined names Column: 1 Error code: F403

    reviewbotreviewbot

    'setuptools.build_meta.*' imported but unused Column: 1 Error code: F401

    reviewbotreviewbot

    Was this supposed to be included here alongside the build types?

    maubinmaubin

    This is meant to be tool.buildthings.editable.extra-build-steps

    maubinmaubin

    Typo "doen't".

    maubinmaubin

    This should just be [tool.buildthings.isolation].

    maubinmaubin

    Same here, we can remove the word "editable" since this function is used with any build type.

    maubinmaubin

    This is missing in the Args section of the docstring.

    maubinmaubin

    Do we wanna use logger here instead of a print?

    maubinmaubin

    This needs a Raises section in the docs.

    maubinmaubin

    This needs a Raises docs section too.

    maubinmaubin

    This needs a Raises docs section too.

    maubinmaubin

    exclude_deps contains package names, but deps has full dependency specifiers. We probably need to parse the items in deps and …

    daviddavid

    This shouldn't talk about Djblets and Review Board.

    daviddavid

    For now this is okay, but keying off only the leaf node might bite us in the future. Is there …

    daviddavid

    Can we document in here the stuff about different build types having different local packages, but that's okay because the …

    daviddavid

    This will quietly swallow requirements lines like -r some-other-file.txt, -e ., or --index-url .... We probably want to document that …

    daviddavid
    Checks run (1 failed, 1 succeeded)
    flake8 failed.
    JSHint passed.

    flake8

    chipx86
    chipx86
    david
    1. 
        
    2. Show all issues

      We need to add toml as a dependency for python_version<3.11

    3. Show all issues

      There's no override of prepare_metadata_for_build_sdist. Should we have one?

      1. That's not one of the build backend hooks. Metadata's only for wheels, as it goes into the .dist-info directory.

    4. README.rst (Diff revision 1)
       
       
       
      Show all issues

      This should probably have a line of ===== above it too.

    5. buildthings/backend.py (Diff revision 1)
       
       
       
       
       
       
       
       
       
       
       
       
      Show all issues

      Looks like some of this might have been copy/pasted from sdist

    6. buildthings/backend.py (Diff revision 1)
       
       
       
       
      Show all issues

      Are these things actually necessary for the metadata operation? This is pretty heavyweight, and seems like it could be limited to run only in the actual build_* hooks.

    7. buildthings/backend.py (Diff revision 1)
       
       
      Show all issues

      The local_packages_path in isolation_config is str | None but apply_local_dep_paths types this as str

    8. buildthings/backend.py (Diff revision 1)
       
       
       
      Show all issues

      This will end up computing npm_workspaces_dir relative to buildthings/backend.py. I think we want the cwd instead.

    9. buildthings/backend.py (Diff revision 1)
       
       
      Show all issues

      Because command is an str, this needs shell=True

    10. buildthings/backend.py (Diff revision 1)
       
       
       
       
       
       
       
       
      Show all issues

      We should pass text=True to subprocess.run(), otherwise the error will look like this:

      Failed to install npm packages: b'...'

    11. buildthings/config.py (Diff revision 1)
       
       
      Show all issues

      Shouldn't this be ImportError?

    12. buildthings/local_paths.py (Diff revision 1)
       
       
       
       
      Show all issues

      This cache works when we're doing a single type of build, but in the case where a package has different local_packages for different targets, we could mess up (for example, python -m build does both wheel and sdist by default, I believe without reloading the build backend). I think we need to have this key off the build type as well.

      1. We discussed this a bit elsewhere, but basically, PEP 517 states states that the build environment should run each hook in a fresh subprocess in order to allow the build backend to set environment variables and manage other global state without risk of conflict between hooks and build types.

        This is codified in pyproject_hooks, which is the de facto way of interfacing with build backends.

        I can verify that this is the case, as each hook is run with its own PID and command line arguments.

        It's possible there are bad implementations out there, but not ones we will be using or recommending.

    13. buildthings/requirements.py (Diff revision 1)
       
       
       
       
       
      Show all issues

      This drops invalid requirements lines silently.

    14. 
        
    chipx86
    Review request changed
    Change Summary:
    • Added toml as a dependency for Python < 3.11.
    • Added logging when encountering an invalid requirement line in a requirements.txt-type file.
    • Removed npm management during prepare_metadata_for_build_wheel().
    • Passed shell=True when running build steps.
    • Passed text=True when running npm install, to help with error reporting.
    • Fixed some doc issues.
    • Fixed apply_local_dep_paths to take a None local_packages_path.
    • Fixed the npm_workspaces_dir to be based on the current working directory of the tree.
    • Fixed an IOError that should have been ImportError.
    Commits:
    Summary ID
    Add the buildthings build backend.
    Buildthings is a setuptools-based Python build backend that makes it easier to work with projects with more complex build requirements, particularly those involving NPM or packages that may still be in development in local source trees. It's based on the build backends we introduced in Review Board and Djblets, but more refined and capable of being expanded for additional use cases down the road. This initial version supports: * `.local-packages/`, mapping dependency names to local source trees, used in isolated build environments (available per-build type). * Options for including dev- and install-time dependencies within the isolated build environments (available per-build type). * An option for excluding certain, more complex dependencies from the isolated environments (available per-build type). * Options for enabling NPM installs and for managing workspaces based on other Python packages. * Support for extra build steps for editables, wheels, and source distributions. Much of this behaves the way it did in our in-tree build backends, but with several behavior issues fixed. For example, development dependencies will now map to local paths if found in `.local-packages/`. Further, depending on an editable package using this backend will no longer error out when it in turn depends on other local trees. Full documentation for all the options are available in the README.
    4027a0f9a480988426a486cea07259604217ef5d
    Add the buildthings build backend.
    Buildthings is a setuptools-based Python build backend that makes it easier to work with projects with more complex build requirements, particularly those involving NPM or packages that may still be in development in local source trees. It's based on the build backends we introduced in Review Board and Djblets, but more refined and capable of being expanded for additional use cases down the road. This initial version supports: * `.local-packages/`, mapping dependency names to local source trees, used in isolated build environments (available per-build type). * Options for including dev- and install-time dependencies within the isolated build environments (available per-build type). * An option for excluding certain, more complex dependencies from the isolated environments (available per-build type). * Options for enabling NPM installs and for managing workspaces based on other Python packages. * Support for extra build steps for editables, wheels, and source distributions. Much of this behaves the way it did in our in-tree build backends, but with several behavior issues fixed. For example, development dependencies will now map to local paths if found in `.local-packages/`. Further, depending on an editable package using this backend will no longer error out when it in turn depends on other local trees. Full documentation for all the options are available in the README.
    316b1b344bd10a11858b4c6e91671dabb102d409

    Checks run (1 failed, 1 succeeded)

    flake8 failed.
    JSHint passed.

    flake8

    david
    1. 
        
    2. buildthings/requirements.py (Diff revisions 1 - 2)
       
       
       
       
       
       
       
       
       
       
       

      Oh oh oh! TIL:

      return [
          parsed
          for line in fp
          if (parsed := _parse_requirement_line(line, path))
      ]
      
    3. pyproject.toml (Diff revisions 1 - 2)
       
       
      Show all issues

      This should be able to just be < instead of <=

    4. 
        
    chipx86
    maubin
    1. 
        
    2. README.rst (Diff revision 3)
       
       
      Show all issues

      Was this supposed to be included here alongside the build types?

    3. README.rst (Diff revision 3)
       
       
      Show all issues

      This is meant to be tool.buildthings.editable.extra-build-steps

    4. buildthings/backend.py (Diff revision 3)
       
       
      Show all issues

      Typo "doen't".

    5. buildthings/backend.py (Diff revision 3)
       
       
      Show all issues

      This should just be [tool.buildthings.isolation].

    6. buildthings/backend.py (Diff revision 3)
       
       
      Show all issues

      Same here, we can remove the word "editable" since this function is used with any build type.

    7. buildthings/backend.py (Diff revision 3)
       
       
      Show all issues

      This is missing in the Args section of the docstring.

    8. buildthings/backend.py (Diff revision 3)
       
       
      Show all issues

      Do we wanna use logger here instead of a print?

      1. This is a good question. There doesn't appear to be a standard approach. I'm seeing setuptools using both. I think a log functino is fine, though, so I'll switch to that.

    9. buildthings/config.py (Diff revision 3)
       
       
      Show all issues

      This needs a Raises section in the docs.

    10. buildthings/config.py (Diff revision 3)
       
       
      Show all issues

      This needs a Raises docs section too.

    11. buildthings/config.py (Diff revision 3)
       
       
      Show all issues

      This needs a Raises docs section too.

    12. 
        
    chipx86
    david
    1. 
        
    2. buildthings/backend.py (Diff revision 4)
       
       
       
       
       
       
       
      Show all issues

      exclude_deps contains package names, but deps has full dependency specifiers. We probably need to parse the items in deps and canonicalize both deps and exclude_deps names before comparing them.

      1. I'm going to punt on this for right now. I'll circle back to it. This release is now a showstopper and this limitation can be worked around by specifying the full specifier.

      2. Can you add a TODO comment about this?

    3. buildthings/backend.py (Diff revision 4)
       
       
       
       
      Show all issues

      This shouldn't talk about Djblets and Review Board.

    4. buildthings/config.py (Diff revision 4)
       
       
      Show all issues

      For now this is okay, but keying off only the leaf node might bite us in the future. Is there any reason to not use the full key in self.dynamic?

      1. This is how this works in setuptools and other build backends. It's a reference to leaf keys that are handled specifically by the build backend in specific sections.

    5. buildthings/local_paths.py (Diff revision 4)
       
       
      Show all issues

      Can we document in here the stuff about different build types having different local packages, but that's okay because the build backend gets reloaded?

    6. buildthings/requirements.py (Diff revision 4)
       
       
       
       
       
       
       
      Show all issues

      This will quietly swallow requirements lines like -r some-other-file.txt, -e ., or --index-url .... We probably want to document that limitation in here.

      1. Same limitation that setuptools has. I'll make a note.

    7. 
        
    chipx86
    david
    1. Ship It!
    2. 
        
    maubin
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to master (292cc85)