Speed up search indexing.

Review Request #5935 — Created June 5, 2014 and submitted

Information

Review Board
release-2.0.x
b1500e5...

Reviewers

Search indexing was painfully slow, since every single ReviewRequest
being indexed ended up adding one query for the submitter and one to get
a list of all FileDiffs.

The FileDiffs query in particular resulted in quite a lot of unnecessary
time spent.

By taking advantage of select_related(), prefetch_related(), and a
prepare_file() statement, we can reduce this down to only 2 additional
queries per batch of review requests (as defined by Haystack, defaulting
to 1 large batch).

Ran an index against a local MySQL with 2,572 review requests, and tested that
file queries still worked as expected.

Added some debugging to the update_index command to see the resulting query
counts.

Before this change, there were 7,724 queries for 2,572 review requests.

After this change, there were 14.

Description From Last Updated

'FileDiff' imported but unused

reviewbotreviewbot
reviewbot
  1. This is a review from Review Bot.
      Tool: PEP8 Style Checker
      Processed Files:
        reviewboard/reviews/search_indexes.py
        reviewboard/reviews/models/review_request.py
      Ignored Files:
    
    
  2. 
      
reviewbot
  1. This is a review from Review Bot.
      Tool: Pyflakes
      Processed Files:
        reviewboard/reviews/search_indexes.py
        reviewboard/reviews/models/review_request.py
      Ignored Files:
    
    
  2. Show all issues
     'FileDiff' imported but unused
    
  3. 
      
chipx86
reviewbot
  1. This is a review from Review Bot.
      Tool: PEP8 Style Checker
      Processed Files:
        reviewboard/reviews/search_indexes.py
        reviewboard/reviews/models/review_request.py
      Ignored Files:
    
    
  2. 
      
reviewbot
  1. This is a review from Review Bot.
      Tool: Pyflakes
      Processed Files:
        reviewboard/reviews/search_indexes.py
        reviewboard/reviews/models/review_request.py
      Ignored Files:
    
    
  2. 
      
david
  1. Ship It!

  2. 
      
chipx86
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-2.0.x (c969a22)
Loading...