Fix thread safety issues with the forwarding cache backend.
Review Request #6644 — Created Nov. 25, 2014 and submitted
The forwarding cache backend had a thread safety problem where a thread would be resetting the cache while another was attempting to access it, resulting in the access being performed on a None value. This is a problem because in Django, a cache backend is shared across all threads. A couple things were done to improve the thread safety. First, we no longer reset the backend to None. Instead, we replace the old backend with a new one. Accesses to the old backend may still occur without completely breaking. Second, we reduce the chance of such collisions by preventing several threads from reloading the backend at the same time. We now keep a generation number for the cache backend. When we go to load a new backend, the thread will locally store the last seen generation number and then attempt to grab a lock. Whichever thread acquires that lock first will set the new backend and then update the generation number. Subsequent threads that grab the lock will check their local copy of the old generation number against the stored one, and use that to decide whether to load the backend.
I've been dealing with random HTTP 500s in Chrome and sometimes in Firefox
for a while now. Chrome showed no payload, and I thought this was another
case of the browser trying to fetch more requests at once than the server
could handle (an old Chrome bug that was fixed at one point).Digging into this, I found there were cache backend accesses against None,
and I found that it was due to threads stomping over each other.This fix solved it. I've reloaded in Chrome and Firefox dozens of times
without a single HTTP 500 error.Unit tests also pass.