• 
      

    Add a condensediffs management command for condensing stored diff data.

    Review Request #5138 — Created Dec. 18, 2013 and submitted

    Information

    Review Board
    master

    Reviewers

    Add a condensediffs management command for condensing stored diff data.

    Review Board 1.7 introduced deduping of new diffs, but older diffs
    remained duplicated in the database. This would add up, particularly
    when using parent diffs or when uploading multiple revisions of diffs
    containing many files unchanged between revisions.

    This introduces a condensediffs command that performs the deduping logic
    for all older diffs. This operation is safe to perform while the
    database is running, and should be safe to terminate if needed. The
    actual logic for all this is in the new FileDiffManager.

    Upon completion, the command will display the amount of bytes saved as a
    before/after comparison and as a percentage of reduction.

    rb-site upgrade will recommend that users run this if they have any
    unmigrated data as well.

    This will be backported to 1.7.x, after the migrate-on-access code is
    backported.

    Ran this against my test database and a copy of the reviews.reviewboard.org
    database. The migrations were successful and the review requests seemed fine.

    Description From Last Updated

    Instead of using print(), management commands should use self.stdout.write(). Is this command safe to ctrl+C?

    david david

    self.stdout.write()

    david david

    While the exclamation point is fun, it might look really bad if it shouts out (1% savings!).

    david david
    david
    1. 
        
    2. reviewboard/diffviewer/management/commands/condensediffs.py (Diff revision 1)
       
       
       
       
       
       
       
      Show all issues

      Instead of using print(), management commands should use self.stdout.write().

      Is this command safe to ctrl+C?

      1. It should be, as these are both in a transaction and the data is migrated in a way that should ensure no data is lost (new table is created if it doesn't exist with the new data, then saved, then data is removed from FileDiff, then saved). If it's stopped somewhere in there, then the state should still be fine for anybody who views the diffs, or for the next transaction.

        That said, something potentially could go wrong, maybe.

    3. Show all issues

      self.stdout.write()

    4. Show all issues

      While the exclamation point is fun, it might look really bad if it shouts out (1% savings!).

    5. 
        
    chipx86
    chipx86
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed