Add formal support for writing both simulation and integration tests.

Review Request #11555 — Created March 26, 2021 and submitted

chipx86
ReviewBot
release-3.0.x
reviewbot

To ensure Review Bot works as expected, we need to write tests that
simulate a call to an external tool, and tests that actually execute an
external tool. A few of these have already been written, but maintenance
was already becoming problematic, as we want to ensure consistency
in both the data going into these tests and the data we assert.

This adds a new BaseToolTestCase class for tool tests, along with an
accompanying ToolTestCaseMetaclass and @integration_test and
@simulation_test decorators.

The decorators mark a test as being capable of integration testing and
simulation testing, respectively. They're intended for
BaseTool.execute() unit tests. The metaclass will scan for these and
split any test function into separate simulation/integration tests (if
the appropriate decorators are applied).

This keeps things very easy to maintain, but with the expense of a quirk
with test names. A test_foo() defined on the class will no longer be
available to the test runner. Instead, separate
test_simulation_foo() and test_integration_foo() test functions will
be available. This isn't immediately obvious, but is a quirk we can
learn to live with.

Made use of this in upcoming changes. Tests all passed.

Summary
Add formal support for writing both simulation and integration tests.
Description From Last Updated

E127 continuation line over-indented for visual indent

reviewbotreviewbot

E127 continuation line over-indented for visual indent

reviewbotreviewbot
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

chipx86
Review request changed

Change Summary:

Fixed an indentation error.

Commits:

Summary
-
Add formal support for writing both simulation and integration tests.
+
Add formal support for writing both simulation and integration tests.

Diff:

Revision 2 (+1388 -1130)

Show changes

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

david
  1. Ship It!
  2. 
      
chipx86
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-3.0.x (c8fcac0)
Loading...