• 
      

    Better manage memory usage of condensediffs.

    Review Request #5870 — Created May 24, 2014 and submitted — Latest diff uploaded

    Information

    Review Board
    release-2.0.x
    eb69fb2...

    Reviewers

    condensediffs had some memory issues. Even though we were using
    QuerySet.iterator, it seems memory was still increasing far too much.

    To work around this, we're using a couple tricks that we used in
    loaddb and dumpdb. We're only operating on batches of 200 diffs at a
    time, and are resetting queries and garbage-collecting after each batch.

    The query resets shouldn't impact production installs, since DEBUG
    should be False, but it's a precaution. We're also forcing DEBUG to be
    False in the management command as well.

    With these changes, memory still increases over time, to a degree, but
    seems to stabilize. This is with a sample set of over 8200 diffs.

    Added some debug information to check process memory usage before and after.

    Before, memory was steadily rising in usage (even with DEBUG = False).

    After, memory rose for a bit and then stayed pretty steady, without growing
    unexpectedly large.