Add a condensediffs management command for condensing stored diff data.

Review Request #5138 — Created Dec. 18, 2013 and submitted

Information

Review Board
master

Reviewers

Add a condensediffs management command for condensing stored diff data.

Review Board 1.7 introduced deduping of new diffs, but older diffs
remained duplicated in the database. This would add up, particularly
when using parent diffs or when uploading multiple revisions of diffs
containing many files unchanged between revisions.

This introduces a condensediffs command that performs the deduping logic
for all older diffs. This operation is safe to perform while the
database is running, and should be safe to terminate if needed. The
actual logic for all this is in the new FileDiffManager.

Upon completion, the command will display the amount of bytes saved as a
before/after comparison and as a percentage of reduction.

rb-site upgrade will recommend that users run this if they have any
unmigrated data as well.

This will be backported to 1.7.x, after the migrate-on-access code is
backported.

Ran this against my test database and a copy of the reviews.reviewboard.org
database. The migrations were successful and the review requests seemed fine.

Description From Last Updated

Instead of using print(), management commands should use self.stdout.write(). Is this command safe to ctrl+C?

daviddavid

self.stdout.write()

daviddavid

While the exclamation point is fun, it might look really bad if it shouts out (1% savings!).

daviddavid
david
  1. 
      
  2. reviewboard/diffviewer/management/commands/condensediffs.py (Diff revision 1)
     
     
     
     
     
     
     
    Show all issues

    Instead of using print(), management commands should use self.stdout.write().

    Is this command safe to ctrl+C?

    1. It should be, as these are both in a transaction and the data is migrated in a way that should ensure no data is lost (new table is created if it doesn't exist with the new data, then saved, then data is removed from FileDiff, then saved). If it's stopped somewhere in there, then the state should still be fine for anybody who views the diffs, or for the next transaction.

      That said, something potentially could go wrong, maybe.

  3. Show all issues

    self.stdout.write()

  4. Show all issues

    While the exclamation point is fun, it might look really bad if it shouts out (1% savings!).

  5. 
      
chipx86
chipx86
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed