Deprecate execute(), allow it to return strings or bytes and make stripping whitespace from the output optional.
Review Request #13283 — Created Sept. 22, 2023 and submitted
Our
rbtools.hooks.common.execute()
function usessubprocess.Popen
and
Popen.communicate
to execute commands and read their output. Between Python 2
and 3,subprocess.Popen(...).communicate()
changed from returning a tuple of
strings to a tuple of bytes. This breaks some of our hook util methods in
rbtools.hooks.git
where we try to perform string operations on the output
ofexecute()
, but we receive bytes instead of strings.Python 3.7 added a
text
parameter toPopen
to control whether file
objects, stdin, stdout, and stderr are opened in text mode or binary mode.
We make use of that parameter by adding our owntext
parameter toexecute()
to control whether to return the stdout output as a string or bytes. This
defaults toFalse
to match the default behavior ofPopen
in Python 3.This also makes it optional to strip whitespace from the output, instead of
always stripping it. This is useful when dealing with commands where we
need to preserve the whitespace in the output, such as withgit diff
.This changes also deprecates the
rbtools.hooks.common.execute()
function
in favour ofrbtools.utils.process.run_process()
, which is more
future-proof and has better type safety.
- Ran into an error when running the git pre-receive hook, which came
from the clash between string and bytes. Applied this fix and ran
the hook again, saw that the error was gone. - Ran unit tests.
Summary | ID |
---|---|
b8e4ba4563a448db38ce518b78da3ebd71691f7b |
Description | From | Last Updated |
---|---|---|
While we're here, let's change this from "Executes and returns" to "Execute and return" |
david | |
Might want to specify here that this only applies when text=True. |
david | |
We should probably use the old behavior by default, where this would be defaulted to falsy (technically None, but False … |
chipx86 | |
Missing , optional. |
chipx86 | |
Missing , optional. |
chipx86 | |
Let's use a PendingRemovalInRBToolsWarning warning. It's less harsh, and avoids a timeline. The reason being that this method is needed … |
chipx86 |
- Commits:
-
Summary ID b94f0f5726dd28b4e0d7f2db9afdaf8d41b88c5f 921420575482c61627cc5bb5e505e8f81bb5e897 - Diff:
-
Revision 2 (+36 -6)
Checks run (2 succeeded)
- Summary:
-
Ensure that the execute method returns strings and not bytes.Allow the execute method to return strings or bytes and make stripping whitespace from the output optional.
- Description:
-
Our
rbtools.hooks.common.execute()
method usessubprocess.Popen
andPopen.communicate
to execute commands and read their output. Between Python 2and 3, subprocess.Popen(...).communicate()
changed from returning a tuple ofstrings to a tuple of bytes. This breaks some of our hook util methods in rbtools.hooks.git
where we try to perform string operations on the outputof execute()
, but we receive bytes instead of strings.Python 3.7 added a
text
parameter toPopen
to control whether fileobjects, stdin, stdout, and stderr are opened in text mode or binary mode. ~ This change sets that parameter to True
so that we can receive strings~ instead of bytes. ~ We make use of that parameter by adding our own text
parameter toexecute()
~ to control whether to return the stdout output as a string or bytes. This + defaults to True
to match the previous behavior.+ + This also makes it optional to strip whitespace from the output, instead of
+ always stripping it. This is useful when dealing with commands where we + need to preserve the whitespace in the output, such as with git diff
. - Commits:
-
Summary ID 921420575482c61627cc5bb5e505e8f81bb5e897 0e4cabeba8b1268d29bf4f4468916dfd42b0a722 - Diff:
-
Revision 3 (+86 -8)
Checks run (2 succeeded)
- Commits:
-
Summary ID 0e4cabeba8b1268d29bf4f4468916dfd42b0a722 0d97af01419d46f88f103a45fd8368e58c092c6e - Diff:
-
Revision 4 (+88 -8)
Checks run (2 succeeded)
-
This change looks fine, but we should probably deprecate this method in favor of the modern
run_process()
code. We'll still need to keep this method around for existing hook scripts for now, though.
- Change Summary:
-
- Deprecates the
execute
function.
- Deprecates the
- Summary:
-
Allow the execute method to return strings or bytes and make stripping whitespace from the output optional.Deprecate execute(), allow it to return strings or bytes and make stripping whitespace from the output optional.
- Description:
-
~ Our
rbtools.hooks.common.execute()
method usessubprocess.Popen
and~ Our
rbtools.hooks.common.execute()
function usessubprocess.Popen
andPopen.communicate
to execute commands and read their output. Between Python 2and 3, subprocess.Popen(...).communicate()
changed from returning a tuple ofstrings to a tuple of bytes. This breaks some of our hook util methods in rbtools.hooks.git
where we try to perform string operations on the outputof execute()
, but we receive bytes instead of strings.Python 3.7 added a
text
parameter toPopen
to control whether fileobjects, stdin, stdout, and stderr are opened in text mode or binary mode. We make use of that parameter by adding our own text
parameter toexecute()
to control whether to return the stdout output as a string or bytes. This defaults to True
to match the previous behavior.This also makes it optional to strip whitespace from the output, instead of
always stripping it. This is useful when dealing with commands where we need to preserve the whitespace in the output, such as with git diff
.+ + This changes also deprecates the
rbtools.hooks.common.execute()
function+ in favour of rbtools.utils.process.run_process()
, which is more+ future-proof and has better type safety. - Commits:
-
Summary ID 0d97af01419d46f88f103a45fd8368e58c092c6e 424c2ac935f8cbca0dbf919ce4f2b2992739c159 - Diff:
-
Revision 5 (+106 -8)
Checks run (2 succeeded)
-
-
We should probably use the old behavior by default, where this would be defaulted to falsy (technically
None
, butFalse
should be fine).Popen()
defaults to binary streams. -
-
-
Let's use a
PendingRemovalInRBToolsWarning
warning. It's less harsh, and avoids a timeline. The reason being that this method is needed for existing hooks being used in the wild today, and until we have a full new hook implementation and can get people onto it, we should avoid telling them to make changes to their hooks.Hooks also have to be careful with stdout/stderr, as that can signal things to some SCM command line tools or servers.
- Description:
-
Our
rbtools.hooks.common.execute()
function usessubprocess.Popen
andPopen.communicate
to execute commands and read their output. Between Python 2and 3, subprocess.Popen(...).communicate()
changed from returning a tuple ofstrings to a tuple of bytes. This breaks some of our hook util methods in rbtools.hooks.git
where we try to perform string operations on the outputof execute()
, but we receive bytes instead of strings.Python 3.7 added a
text
parameter toPopen
to control whether fileobjects, stdin, stdout, and stderr are opened in text mode or binary mode. We make use of that parameter by adding our own text
parameter toexecute()
to control whether to return the stdout output as a string or bytes. This ~ defaults to True
to match the previous behavior.~ defaults to False
to match the default behavior ofPopen
in Python 3.This also makes it optional to strip whitespace from the output, instead of
always stripping it. This is useful when dealing with commands where we need to preserve the whitespace in the output, such as with git diff
.This changes also deprecates the
rbtools.hooks.common.execute()
functionin favour of rbtools.utils.process.run_process()
, which is morefuture-proof and has better type safety. - Commits:
-
Summary ID 424c2ac935f8cbca0dbf919ce4f2b2992739c159 b8e4ba4563a448db38ce518b78da3ebd71691f7b - Diff:
-
Revision 6 (+104 -8)