Summary

Fix a test ordering and state leak issues affecting tool/logging tests.

Review Request #12399 — Created June 21, 2022 and submitted June 22, 2022, 3:50 p.m.

Information

Owner

chipx86

Repository

Review Board

Branch

release-5.0.x

Bugs

Depends On

Reviewers

Groups

reviewboard

People

Description

Depending on the order in which tests were run, the new Tool conflict
tests in test_registry would fail. One failure had to do with the
presence or lack of Tool objects in the database, and the other had to
do with an assertLogs() failing to catch logged warnings.

There were three issues behind these. One was a failure in the test, and
two were state consistency issues:

The new unit test needed Tool objects but failed to use the proper
fixture. This wasn't caught initially due to the next issue.
Tool objects were sometimes populated when not needed by a unit
test, and could be populated at different levels of test setup,
causing state inconsistency depending on which tests were run first.
The solution to this is to avoid automatically populating Tool
objects in SCMToolRegistry when running unit tests, allowing the
previous fixture behavior to work as it did before, and to populate
at the right times.
A unit test in test_review.py globally disabled warnings. I can't
determine why it did this, but it was likely for very old historical
reasons. This affected subsequent tests. This code has now been
removed.

Testing Done

Tested full test runs and various combinations of suites. All the ones
that failed before pass now.

Commits

Summary	ID
Fix a test ordering and state leak issues affecting tool/logging tests. Depending on the order in which tests were run, the new `Tool` conflict tests in `test_registry` would fail. One failure had to do with the presence or lack of `Tool` objects in the database, and the other had to do with an `assertLogs()` failing to catch logged warnings. There were three issues behind these. One was a failure in the test, and two were state consistency issues: 1. The new unit test needed `Tool` objects but failed to use the proper fixture. This wasn't caught initially due to the next issue. 2. `Tool` objects were sometimes populated when not needed by a unit test, and could be populated at different levels of test setup, causing state inconsistency depending on which tests were run first. The solution to this is to avoid automatically populating `Tool` objects in `SCMToolRegistry` when running unit tests, allowing the previous fixture behavior to work as it did before, and to populate at the right times. 3. A unit test in `test_review.py` globally disabled warnings. I can't determine why it did this, but it was likely for very old historical reasons. This affected subsequent tests. This code has now been removed.	667a7674f949867e636a9e26a01c5a2ddc3762ef

Summary

Fix a test ordering and state leak issues affecting tool/logging tests.

Depending on the order in which tests were run, the new `Tool` conflict tests in `test_registry` would fail. One failure had to do with the presence or lack of `Tool` objects in the database, and the other had to do with an `assertLogs()` failing to catch logged warnings. There were three issues behind these. One was a failure in the test, and two were state consistency issues: 1. The new unit test needed `Tool` objects but failed to use the proper fixture. This wasn't caught initially due to the next issue. 2. `Tool` objects were sometimes populated when not needed by a unit test, and could be populated at different levels of test setup, causing state inconsistency depending on which tests were run first. The solution to this is to avoid automatically populating `Tool` objects in `SCMToolRegistry` when running unit tests, allowing the previous fixture behavior to work as it did before, and to populate at the right times. 3. A unit test in `test_review.py` globally disabled warnings. I can't determine why it did this, but it was likely for very old historical reasons. This affected subsequent tests. This code has now been removed.

667a7674f949867e636a9e26a01c5a2ddc3762ef

Issues

Description	From	Last Updated
So the use of @add_fixtures(['test_scmtools']) and fixtures = ['test_scmtools'] work as they did before? Does this mean they are no …	maubin	June 22, 2022, 3:49 p.m.

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

The issue has been dropped. Show all issues

So the use of @add_fixtures(['test_scmtools']) and fixtures = ['test_scmtools'] work as they did before? Does this mean they are no longer deprecated and that they need to be present to use methods like create_repository

chipx86 June 22, 2022, 3:49 p.m.

Usage of test_scmtools will ensure that the database contains Tools, so we'll still need to use them for any tests that in any way depend on a Tool. We should require this less and less, as we rely more on SCMTool IDs. This particular test is all about Tool-based conflicts, so it needed it, but it was missed before due to the state leakage.

Status:: Completed
Change Summary:: Pushed to release-5.0.x (9f51ffe)