• 
      

    Rewrite argument parsing in rb-site to use Python's argparse module.

    Review Request #11369 — Created Jan. 12, 2021 and submitted

    Information

    Review Board
    release-4.0.x

    Reviewers

    rb-site has historically used the legacy optparse module, which had a
    number of drawbacks:

    • Options were defined in a global pool, so they could only be defined
      by a single command.

    • Options for one command were therefore "allowed" by a second command.

    • When running management commands special -- option was required to
      separate rb-site's arguments from the management command's.

    • There wasn't a good way to define per-command help.

    This change addresses all this by moving to argparse and redoing a bit
    of our command handling. Now, each command is its own sub-parser with
    its own help text and arguments. Collision is no longer possible, and
    each command can be considered fully independent.

    The -- argument is no longer needed when running management commands,
    as argparse allows us to capture any remaining options and pass them
    through.

    However, argparse is not perfect, and we had to work around some
    issues. For instance, multi-paragraph help text is not doable by
    default, so we had to apply some custom logic (overriding "private"
    functions that haven't changed since it was introduced) to format text
    ourselves.

    Another gotcha is that they recommend using nargs='+' to capture one
    or more unknown arguments (for our management command), but it's
    actually pretty incompatible with sub-parsers. Instead, we have to use
    an undocumented option, nargs=argparse.PARSER, to get the desired
    behavior. Despite being undocumented, it forms the basis of sub-parsers,
    does exactly what we need, and is heavily referenced in bug reports as a
    better (yet undocumented) option for this purpose.

    While there are things left to be desired, this is a far more
    future-proof way of performing argument parsing, and provides a
    foundation for further improvements.

    Unit tests pass on all supported versions of Python.

    Manually ran each command with all the variations in configuration that
    made sense (such as upgrade with/without a site directory or -all-sites,
    and manage with all sorts of management commands and options).

    Repeated those tests for Python 2.7, 3.6, and 3.9, to check for any
    regressions in behavior across versions.

    Summary ID
    Rewrite argument parsing in rb-site to use Python's argparse module.
    rb-site has historically used the legacy `optparse` module, which had a number of drawbacks: * Options were defined in a global pool, so they could only be defined by a single command. * Options for one command were therefore "allowed" by a second command. * When running management commands special `--` option was required to separate rb-site's arguments from the management command's. * There wasn't a good way to define per-command help. This change addresses all this by moving to `argparse` and redoing a bit of our command handling. Now, each command is its own sub-parser with its own help text and arguments. Collision is no longer possible, and each command can be considered fully independent. The `--` argument is no longer needed when running management commands, as `argparse` allows us to capture any remaining options and pass them through. However, `argparse` is not perfect, and we had to work around some issues. For instance, multi-paragraph help text is not doable by default, so we had to apply some custom logic (overriding "private" functions that haven't changed since it was introduced) to format text ourselves. Another gotcha is that they recommend using `nargs='+'` to capture one or more unknown arguments (for our management command), but it's actually pretty incompatible with sub-parsers. Instead, we have to use an undocumented option, `nargs=argparse.PARSER`, to get the desired behavior. Despite being undocumented, it forms the basis of sub-parsers, does exactly what we need, and is heavily referenced in bug reports as a better (yet undocumented) option for this purpose. While there are things left to be desired, this is a far more future-proof way of performing argument parsing, and provides a foundation for further improvements.
    a4e558d624bf911b4d9fbe4a18364440dd2b77f6
    Description From Last Updated

    F841 local variable 'old_width' is assigned to but never used

    reviewbotreviewbot

    F811 redefinition of unused 'test_with_help_as_command_option' from line 240

    reviewbotreviewbot
    Checks run (1 failed, 1 succeeded)
    flake8 failed.
    JSHint passed.

    flake8

    chipx86
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-4.0.x (0e38ed7)