Rewrite argument parsing in rb-site to use Python's argparse module.

Review Request #11369 — Created Jan. 12, 2021 and submitted

Information

Review Board
release-4.0.x

Reviewers

rb-site has historically used the legacy optparse module, which had a
number of drawbacks:

  • Options were defined in a global pool, so they could only be defined
    by a single command.

  • Options for one command were therefore "allowed" by a second command.

  • When running management commands special -- option was required to
    separate rb-site's arguments from the management command's.

  • There wasn't a good way to define per-command help.

This change addresses all this by moving to argparse and redoing a bit
of our command handling. Now, each command is its own sub-parser with
its own help text and arguments. Collision is no longer possible, and
each command can be considered fully independent.

The -- argument is no longer needed when running management commands,
as argparse allows us to capture any remaining options and pass them
through.

However, argparse is not perfect, and we had to work around some
issues. For instance, multi-paragraph help text is not doable by
default, so we had to apply some custom logic (overriding "private"
functions that haven't changed since it was introduced) to format text
ourselves.

Another gotcha is that they recommend using nargs='+' to capture one
or more unknown arguments (for our management command), but it's
actually pretty incompatible with sub-parsers. Instead, we have to use
an undocumented option, nargs=argparse.PARSER, to get the desired
behavior. Despite being undocumented, it forms the basis of sub-parsers,
does exactly what we need, and is heavily referenced in bug reports as a
better (yet undocumented) option for this purpose.

While there are things left to be desired, this is a far more
future-proof way of performing argument parsing, and provides a
foundation for further improvements.

Unit tests pass on all supported versions of Python.

Manually ran each command with all the variations in configuration that
made sense (such as upgrade with/without a site directory or -all-sites,
and manage with all sorts of management commands and options).

Repeated those tests for Python 2.7, 3.6, and 3.9, to check for any
regressions in behavior across versions.

Summary ID
Rewrite argument parsing in rb-site to use Python's argparse module.
rb-site has historically used the legacy `optparse` module, which had a number of drawbacks: * Options were defined in a global pool, so they could only be defined by a single command. * Options for one command were therefore "allowed" by a second command. * When running management commands special `--` option was required to separate rb-site's arguments from the management command's. * There wasn't a good way to define per-command help. This change addresses all this by moving to `argparse` and redoing a bit of our command handling. Now, each command is its own sub-parser with its own help text and arguments. Collision is no longer possible, and each command can be considered fully independent. The `--` argument is no longer needed when running management commands, as `argparse` allows us to capture any remaining options and pass them through. However, `argparse` is not perfect, and we had to work around some issues. For instance, multi-paragraph help text is not doable by default, so we had to apply some custom logic (overriding "private" functions that haven't changed since it was introduced) to format text ourselves. Another gotcha is that they recommend using `nargs='+'` to capture one or more unknown arguments (for our management command), but it's actually pretty incompatible with sub-parsers. Instead, we have to use an undocumented option, `nargs=argparse.PARSER`, to get the desired behavior. Despite being undocumented, it forms the basis of sub-parsers, does exactly what we need, and is heavily referenced in bug reports as a better (yet undocumented) option for this purpose. While there are things left to be desired, this is a far more future-proof way of performing argument parsing, and provides a foundation for further improvements.
a4e558d624bf911b4d9fbe4a18364440dd2b77f6
Description From Last Updated

F841 local variable 'old_width' is assigned to but never used

reviewbotreviewbot

F811 redefinition of unused 'test_with_help_as_command_option' from line 240

reviewbotreviewbot
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

chipx86
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-4.0.x (0e38ed7)