Add run_process(), a modern replacement for execute().
Review Request #12564 — Created Aug. 23, 2022 and submitted — Latest diff uploaded
execute()function, used for calling out into other processes and
capturing results, has grown out of control over the years, providing 4
sets of inputs that produce different outputs in different combinations.
An attempt to add type annotations for these combinations quickly grew
out of control, and along with causing numerous type-related regressions
in the codebase over the years, it's finally become time to put
This change introduces the successor,
run_process(). The interface is
like a slimmed-down version of
execute(), allowing basic options like
the working directory and environment to be set, stderr to be redirected
to stdout, expected output encoding, and control over error handling and
There is no control over typing. Instead, this always returns a
RunProcessResultobject, which stores a string version of the command
that was run, the exit code, and byte streams for the standard output
and error output.
From there, a caller can access the raw bytes (via
stderr_bytes.read()), Unicode-decoded content (via
stderr.read()), or line-based versions (via
any of the streams).
Results are always byte strings. Decoding to Unicode strings happens
io.TextIOWrapper(which is what Python uses for
sys.stderr). These wrappers are created only on
While these are all stream objects, they don't stream output from the
program. They always contain the full results from the program. If we
were to add streaming someday for some SCM, we could do this with a new
run_process()and new flags on
execute(), all errors (non-0 exit codes) or selective errors
can be ignored. Unlike
execute(), the caller can still get results
when an error is raised. The exception,
RunProcessError, contains the
results as an attribute (
result), containing all the same information
that would normally be returned. Given that, the support for ignoring
errors is really just a convenience around catching an exception.
This new object has full type safety, so type checkers can make sure
that consumers are processing results correctly.
run_process_exec()to perform the actual execution
of a command. This is a wrapper around
subprocess.run()that takes just
a few flags and returns a tuple of
(exit_code, stdout, stderr). Spies
should connect to this instead of
run_process(), as it's as low-level
as a test would need to go, and the results from this are checked against
the parameters passed to
Logging has been improved to provide better output, distinguishing non-0
exit codes representing failure from those with ignored errors (to help
alleviate concerns when people look at debug logs).
Old Python 2 compatibility code around
subprocesshave been removed,
and new defaults (like
execute()has been updated to wrap
run_process(). This function is
now deprecated, though still widely used in the codebase. Upcoming work
will transition over to
All unit tests pass on Python 3.7-3.11.