Log and display useful information when database upgrades fail.

Review Request #12507 — Created Aug. 5, 2022 and submitted — Latest diff uploaded


Review Board


Every so often, someone encounters a bad database upgrade. This often
comes from in-house modifications, canceled upgrades, or an upgrade from
some particular versions where Django Evolution or Django had some bad
behavior in their SQL or project signature generation.

To help diagnose these issues, this change logs useful information when
either the project signature diff is non-empty (indicating missing
evolution state) or an upgrade fails due to an error. The contents of
the log include:

  • Any useful error messages
  • A project signature diff
  • Lists of applied evolutions and migrations
  • Lists of pending evolutions
  • The current stored signature
  • Timestamps of all stored signatures
  • An information (but not schema) dump of the database

Ideally we'd also log the SQL schema, but that'd require some
per-database logic for correctly running each one's SQL dumping command.
Instead, we'll leave that to something handled during support.

There are also fixes for typos in the output message, and more detail
in the upgrade failure message.

This is something I've had on the backburner for a long time, but a
recent hard-to-diagnose failure in CI helped incentivize building this
logic. Hopefully it will help in the future when customers reach out
with problems.

Simulated different failures, both at signature diff time and upgrade
time. Checked the resulting log file and the output for errors.