• 
      

    Speed up the New Review Request page and cache Tool lookup for fewer queries.

    Review Request #2392 — Created June 4, 2011 and submitted

    Information

    Review Board

    Reviewers

    Speed up the New Review Request page and cache Tool lookup for fewer queries.
    
    The New Review Request page can be very slow with large numbers of
    repositories. We were requesting far more information than needed, as we
    only showed the name, so now we request only the name from the database
    when displaying the repository in the list.
    
    The big slowdown, however, was due to the Tool lookups. We grabbed the
    SCMTool from every repository, which required loading the Tool entry for it
    from the database. If there were 100 Git repositories, this meant we'd have 100
    fetches of the Tool from the database. Clearly too inefficient and unnecessary.
    
    Now we have a small caching layer preventing redundant lookups. Tool
    is associated with a ToolManager that provides a special QuerySet that
    sticks all items fetched through get() into a database. We precompute this
    cache by doing an initial all() if the cache is empty, further reducing
    lookups.
    
    This cache is never cleared, except by restarting the server process.
    However, Tool objects never (or should never) change, except when registered
    during an rb-site upgrade, at which point the server should be restarted
    anyway. We can add intelligent clearing of the cache if need be later, but
    it shouldn't be an issue.
    Ran profiling on the New Review Request page. On my deployment of 20
    repositories, it went from a total of 27 database queries on the page
    to 8. The only Tool query it did was the initial cache population,
    meaning we saved 19 queries (number of repositories - 1).
    
    Unit tests passed.