• 
      

    Optimize the loading of unit test fixtures.

    Review Request #10378 — Created Jan. 11, 2019 and submitted

    Information

    Djblets
    release-1.0.x
    d2b3a94...

    Reviewers

    We have a number of optimizations in place for Django unit test
    fixtures, designed to prevent unnecessary repeated deserialization of
    fixture data. This change improves this further, reducing the work
    needed to save the deserialized data to the database.

    Previously, we'd call save() on every deserialized object for every
    test run. This would perform an UPDATE call (to update a previous
    database instance with the object's ID, to reflect the new state),
    falling back to a new INSERT if the UPDATE didn't find any existing
    rows. This meant, in the worst-case scenario, 2 SQL queries per fixture
    object per unit test.

    We now simply delete any existing entries all at once (if any exist) and
    then proceed to bulk-create the replacements. This saves us all those
    SQL queries, requiring only 2 per model instead of 2 per object per
    model.

    On top of this, we no longer directly assign a list of objects to
    ManyToManyFields. An assignment results in a SELECT, possible
    DELETE, and an INSERT, and this is wasteful when we know we have new
    entries. So instead, we call .add() on the relation so that only an
    INSERT is done.

    In testing against the Review Board unit test suite (of ~4000 unit
    tests), this change resulted in a ~45 second reduction in test time with
    no regressions introduced.

    Djblets and Review Board unit tests pass.

    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-1.0.x (3268c22)