• 
      

    Attempt to handle threading-related issues with extension reloading.

    Review Request #5866 — Created May 22, 2014 and submitted — Latest diff uploaded

    Information

    Djblets
    release-0.8.x
    8891572...

    Reviewers

    We observed some issues in a mod_worker environment where attempts to
    reload extensions would fail, causing state to become corrupted, leading
    to more and more errors. Some of these cases were made more
    bullet-proof, but we didn't fix the core issue.

    This change attempts to prevent what may be causing the issue by
    allowing only one thread to reload the list of extensions at a time.

    We have a lock set up for reloading extensions (in case two users try to
    reload in the admin UI at the same time).

    There's a separate lock in the middleware that's a bit smarter about
    calling load(). It checks if the extension state has expired, tries to
    grab a lock, and then checks again for expiration. The goal there is to
    not have a bunch of threads that are waiting decide they're all going to
    trigger their own reloads.

    Unit tests have been written that hit issues both during uninit and
    during init of extensions. These mirror the errors we've been getting
    on reviews.reviewboard.org with Review Bot. With the fix, these tests
    pass.

    I don't have a real-world repro case yet (we'll see in a couple days on
    reviews.reviewboard.org if we start to hit any problems), but I was able
    to reproduce some of the errors we through unit tests.

    Basically, there's a number of possible failure points, depending on what
    instructions are being executed, but the tests manage to reliably trigger
    one error during init and one during uninit. Those trigger only if I turn off
    locking. Everything passes if locking is on.