Update CargoTool for new tool support and improved parsing.

Review Request #11654 — Created June 12, 2021 and submitted

Information

ReviewBot
release-3.0.x

Reviewers

This updates CargoTool to subclass the newer BaseTool, and to use
the newer features it offers (dependencies, improved file extension
handling, centralized base command building, and new comment options).

The parsing has been completely redone. This tool runs both
cargo clippy and cargo test, and these output content differently.
For cargo clippy, we can safely use --message-format=json to get a
series of JSON payloads we can process, which works well, and is less
error-prone than the previous parsing logic.

However, JSON output is, as of now, only available in nightly builds of
cargo test, so we can't depend on it. Instead, we use pattern matching
to try to find the test output, turning that into a general comment.
It's still better than the old logic, which as far as I can tell no
longer works (or maybe never did), since the old logic was looking for
output that isn't present in at least modern builds of cargo test.

There's also some additional handling for unit tests. If cargo clippy
is enabled, it will be run against unit tests, using the (semi-modern
--tests argument). If not, and cargo test encounters a syntax error,
it will be reported, along with recommendations.

Test output is also capped to 100 lines.

Unit tests cover all these situations, and we have notes to support
future JSON output from cargo clippy.

This is the last tool that needed to be ported over to the new tool
and test support.

Unit tests pass on Python 2.7 and 3.x.

Summary ID
Update CargoTool for new tool support and improved parsing.
This updates `CargoTool` to subclass the newer `BaseTool`, and to use the newer features it offers (dependencies, improved file extension handling, centralized base command building, and new comment options). The parsing has been completely redone. This tool runs both `cargo clippy` and `cargo test`, and these output content differently. For `cargo clippy`, we can safely use `--message-format=json` to get a series of JSON payloads we can process, which works well, and is less error-prone than the previous parsing logic. However, JSON output is, as of now, only available in nightly builds of `cargo test`, so we can't depend on it. Instead, we use pattern matching to try to find the test output, turning that into a general comment. It's still better than the old logic, which as far as I can tell no longer works (or maybe never did), since the old logic was looking for output that isn't present in at least modern builds of `cargo test`. There's also some additional handling for unit tests. If `cargo clippy` is enabled, it will be run against unit tests, using the (semi-modern `--tests` argument). If not, and `cargo test` encounters a syntax error, it will be reported, along with recommendations. Unit tests cover all these situations, and we have notes to support future JSON output from `cargo clippy`.
7315cae7c75e85a3c42cb4364f330d45d94ff3a9
Description From Last Updated

typo: conver -> convert

daviddavid
david
  1. 
      
  2. bot/reviewbot/tools/cargotool.py (Diff revision 1)
     
     
    Show all issues

    typo: conver -> convert

  3. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-3.0.x (f5d81a6)