Summary

Recurse into nested .local-packages when setting up build environments.

Review Request #15103 — Created June 7, 2026 and updated June 7, 2026, 4:34 p.m.

Information

Owner

chipx86

Repository

buildthings

Branch

master

Bugs

Depends On

Reviewers

Groups

beanbag-misc

People

Description

When setting up the isolated build environment, buildthings would look
up the .local-packages and replace any mentions of dependencies with
those paths.

The problem came from when one of those dependencies had its own
.local-packages. Those would also be found at that package's own build
time, but the resulting wheel used for the parent's isolated build
environment wouldn't contain those replaced dependencies. This would
cause the main build environment to fail, since it had no knowledge of
the paths.

Now, buildthings will recurse into .local-packages from any package
mentioned in a .local-packages and mark them as explicit build-time
dependencies. This differs from the normal approach of applying those to
a list of dependencies, since we don't have the ability to intercept and
normalize any dependencies' dependencies. So we add them to the
build-time dependencies to ensure they get installed and satisfy the
packages.

Package-time dependencies are not affected.

Testing Done

Unit tests pass.

Successfully installed Review Board 9, which had a local-packages
dependency chain looking like:

reviewboard -> Djblets, django_evolution
Djblets -> django_evolution, django-assert-queries

This was failing to install django-assert-queries prior to this
change. With the fix, it now finds and installs it in the build
environment successfully.

Commits

Summary	ID
Recurse into nested .local-packages when setting up build environments. When setting up the isolated build environment, buildthings would look up the `.local-packages` and replace any mentions of dependencies with those paths. The problem came from when one of those dependencies had its own `.local-packages`. Those would also be found at that package's own build time, but the resulting wheel used for the parent's isolated build environment wouldn't contain those replaced dependencies. This would cause the main build environment to fail, since it had no knowledge of the paths. Now, buildthings will recurse into `.local-packages` from any package mentioned in a `.local-packages` and mark them as explicit build-time dependencies. This differs from the normal approach of applying those to a list of dependencies, since we don't have the ability to intercept and normalize any dependencies' dependencies. So we add them to the build-time dependencies to ensure they get installed and satisfy the packages. Package-time dependencies are not affected.	29d3de55ad492e09382d98405e02183e07f0ec64

Summary

Recurse into nested .local-packages when setting up build environments.

When setting up the isolated build environment, buildthings would look up the `.local-packages` and replace any mentions of dependencies with those paths. The problem came from when one of those dependencies had its own `.local-packages`. Those would also be found at that package's own build time, but the resulting wheel used for the parent's isolated build environment wouldn't contain those replaced dependencies. This would cause the main build environment to fail, since it had no knowledge of the paths. Now, buildthings will recurse into `.local-packages` from any package mentioned in a `.local-packages` and mark them as explicit build-time dependencies. This differs from the normal approach of applying those to a list of dependencies, since we don't have the ability to intercept and normalize any dependencies' dependencies. So we add them to the build-time dependencies to ensure they get installed and satisfy the packages. Package-time dependencies are not affected.

29d3de55ad492e09382d98405e02183e07f0ec64

Issues

Description	From	Last Updated
While local packages are an editable thing, just for consistency I feel like this should pass the build type in …	david	June 7, 2026, 4:31 p.m.
Can we dedupe by (canonicalized) package name?	david	June 7, 2026, 4:32 p.m.
Even though this is "internal", given that it's not unused, we should call it seen instead of _seen	david	June 7, 2026, 4:24 p.m.
dict -> set	david	June 7, 2026, 4:24 p.m.
Can we add a type annotation here?	david	June 7, 2026, 4:24 p.m.
This method has some pretty intense side effects, prepending a bunch of stuff to sys.path and resolving dynamic keys (importing …	david	June 7, 2026, 4:24 p.m.

flake8 passed.

JSHint passed.

Fix it!

buildthings/backend.py (Diff revision 1)

The issue has been dropped. Show all issues

While local packages are an editable thing, just for consistency I feel like this should pass the build type in (since we have it).

chipx86

June 7, 2026, 4:34 p.m.

That'd be the wrong value, because build types don't match up in this way. The isolated build environment is installing local dependencies explicitly as editable wheels, no matter the parent package's build type, and we need to be following that.

buildthings/backend.py (Diff revision 1)

The issue has been dropped. Show all issues

Can we dedupe by (canonicalized) package name?

chipx86

June 7, 2026, 4:34 p.m.

I was going to work on that but it ultimately doesn't matter. It's a lot of work for no gain. The package manager will sort it out at isolated build env install time.

buildthings/local_paths.py (Diff revision 1)

An issue was opened. Show all issues

Even though this is "internal", given that it's not unused, we should call it seen instead of _seen

chipx86

June 7, 2026, 4:34 p.m.

It's only used for recursion into itself. I don't want it to be seen as part of the caller's responsibility in any way. Ideally it wouldn't even be in the function signature, but I didn't feel it was worth adding a nested function inside (which this started out as, and I collapsed it down).

buildthings/local_paths.py (Diff revision 1)
An issue was opened. Show all issues
```
dict -> set
```
buildthings/local_paths.py (Diff revision 1)
An issue was opened. Show all issues
```
Can we add a type annotation here?
```

buildthings/local_paths.py (Diff revision 1)

An issue was opened. Show all issues

This method has some pretty intense side effects, prepending a bunch of stuff to sys.path and resolving dynamic keys (importing those modules).

Since we just care about the local_packages_path key, can we load the toml file directly and read that key?

chipx86

June 7, 2026, 4:34 p.m.

No because of the way that this computes defaults, but I'll follow up this change with one that creates a more side-effect-free loading process. That'll be a big enough change where I don't want to fold it into this.