Add a management command to clean up orphaned data.

Review Request #13740 — Created April 17, 2024 and submitted

Information

Review Board
release-7.x

Reviewers

This change adds a new management command that will clean up various
pieces of data in the database and filesystem that could have been left
behind if administrators used the "Delete Permanently" action on a
review request or deleted review requests via the admin UI.

It will query each of the affected items to get a total count, and then
perform the actual deletion in batches. This allows us to show nice
progress bars, as well as make the command interruptible.

Populated my database with a bunch of data including screenshots,
file attachments, diffs, and change descriptions. Temporarily disabled
our new signal handler and deleted the review requests.

  • Ran with --show-counts-only and saw the expected counts for each type
    of object.
  • Ran normally and watched it delete all the objects.
  • Ran again with --show-counts-only and saw that it no longer reported
    any orphaned data.
  • Set the batch size low and verified that the progress bars worked
    correctly.
  • Built and looked at the docs.
Summary ID
Add a management command to clean up orphaned data.
This change adds a new management command that will clean up various pieces of data in the database and filesystem that could have been left behind if administrators used the "Delete Permanently" action on a review request or deleted review requests via the admin UI. It will query each of the affected items to get a total count, and then perform the actual deletion in batches. This allows us to show nice progress bars, as well as make the command interruptible. Testing Done: - Populated my database with a bunch of data including screenshots, file attachments, diffs, and change descriptions. Temporarily disabled our new signal handler and deleted the review requests. Ran with --show-counts-only and saw the expected counts for each type of object. Ran normally and watched it delete all the objects. Ran again with --show-counts-only and saw that it no longer reported any orphaned data. - Set the batch size low and verified that the progress bars worked correctly. - Built and looked at the docs.
e45b6c7f2d5c4d59c0c67d381f2cbf62dffef994
Description From Last Updated

continuation line over-indented for visual indent Column: 21 Error code: E127

reviewbotreviewbot

We should go with "data" or "objects" here instead of "file attachments" to be more general. Same with the module …

maubinmaubin

This should be taken out.

maubinmaubin

This is missing a ::.

chipx86chipx86

I think this needs to be gettext_lazy.

chipx86chipx86

Small nit: One per line would help keep this readable/more maintainable if we need to expand upon this.

chipx86chipx86

Same here.

chipx86chipx86

Keyword-only arguments would help make this a bit more readable.

chipx86chipx86
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

david
maubin
  1. 
      
  2. reviewboard/reviews/management/commands/clean-orphaned-data.py (Diff revision 2)
     
     
     
     
     
     
     
     
    Show all issues

    We should go with "data" or "objects" here instead of "file attachments" to be more general. Same with the module docstring too.

    1. Leftovers from before I discovered that we had more than just file attachments.

  3. Show all issues

    This should be taken out.

  4. 
      
david
maubin
  1. Ship It!
  2. 
      
chipx86
  1. This looks good.

    Some small suggestions to help with readability, but overall looks good.

    As a suggestion for a possible feature, it might be nice to be able to get a list of the IDs for each type of object that's going to be deleted so it can be audited or manually dealt with prior to actually invoking a deletion. But not a blocker for this change by any means.

  2. Show all issues

    This is missing a ::.

  3. Show all issues

    I think this needs to be gettext_lazy.

  4. Show all issues

    Small nit: One per line would help keep this readable/more maintainable if we need to expand upon this.

  5. Show all issues

    Same here.

  6. Show all issues

    Keyword-only arguments would help make this a bit more readable.

  7. 
      
david
david
Review request changed
Status:
Completed
Change Summary:
Pushed to release-7.x (e2b13e6)