• 
      

    Fix thread safety issues with the forwarding cache backend.

    Review Request #6644 — Created Nov. 25, 2014 and submitted

    Information

    Djblets
    release-0.8.x
    4e037d0...

    Reviewers

    The forwarding cache backend had a thread safety problem where a thread
    would be resetting the cache while another was attempting to access it,
    resulting in the access being performed on a None value. This is a
    problem because in Django, a cache backend is shared across all threads.
    
    A couple things were done to improve the thread safety.
    
    First, we no longer reset the backend to None. Instead, we replace the
    old backend with a new one. Accesses to the old backend may still occur
    without completely breaking.
    
    Second, we reduce the chance of such collisions by preventing several
    threads from reloading the backend at the same time. We now keep a
    generation number for the cache backend. When we go to load a new
    backend, the thread will locally store the last seen generation number
    and then attempt to grab a lock.
    
    Whichever thread acquires that lock first will set the new backend and
    then update the generation number. Subsequent threads that grab the lock
    will check their local copy of the old generation number against the
    stored one, and use that to decide whether to load the backend.

    I've been dealing with random HTTP 500s in Chrome and sometimes in Firefox
    for a while now. Chrome showed no payload, and I thought this was another
    case of the browser trying to fetch more requests at once than the server
    could handle (an old Chrome bug that was fixed at one point).

    Digging into this, I found there were cache backend accesses against None,
    and I found that it was due to threads stomping over each other.

    This fix solved it. I've reloaded in Chrome and Firefox dozens of times
    without a single HTTP 500 error.

    Unit tests also pass.

    reviewbot
    1. Tool: Pyflakes
      Processed Files:
          djblets/cache/forwarding_backend.py
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          djblets/cache/forwarding_backend.py
      
      
    2. 
        
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-0.8.x (9ec8fd2)