Fix issues upgrading SCMTool IDs on non-pristine setups.

Review Request #12380 — Created June 17, 2022 and submitted — Latest diff uploaded


Review Board


While trying to upgrade my system to the new registry-backed SCMTool
support, a number of issues with the new upgrade process for SCMTools
0were discovered, in part due to the complexity and history of my local

These issues include:

  1. Pre-upgrade only recorded state to upgrade if the
    repository_scmtool_id was not yet in the database, preventing a
    re-attempt at upgrading from working on failure (and outright
    skipping the pre-upgrade work if running evolve --execute before
    running a managed upgrade).

  2. SCMTools that could not be loaded (missing package) would cause the
    pre-upgrade steps to crash and no IDs to be migrated.

  3. Integration configurations that lacked a saved conditions field
    would cause a lookup on conditions to try to load the integration
    class in order to retrieve the default, and as these are provided by
    unloaded extensions, this would cause the pre-upgrade steps to

This change fixes all three scenarios.

Pre-upgrade steps for SCMTools are now run if either the evolution is
missing from the database or if we can find any repositories lacking
an ID. This enables re-attempts in both the premature evolve --execute
case and in the upgrade re-attempt case.

We now check for exceptions on getting the SCMTool IDs. If we fail to
load any, we'll list them and follow up with some helpful instructions.
These repositories will remain in their pre-upgrade state until resolved
(an upcoming change will improve upon this bad state).

For updates to integration configuration, the code is now careful to
work directly with the settings dictionary rather than the wrapper
methods, to avoid an attempted load of the integration. As an
optimization, the queryset also avoids loading any more state than
needed for the upgrade.

Tested an upgrade on a clean database.

Tested an upgrade on my messy database, with missing SCMTools and
integration configurations both without explicit conditions saved
and without a ConditionsField.

Verified that the IDs got migrated over even when errors were

After resolving the errors, I verified that the upgrade could be
re-attempted and the remaining SCMTool IDs were looked up and
migrated over.