Add support for Elasticsearch

Review Request #8209 — Created June 2, 2016 and submitted

Information

Review Board
release-3.0.x
2a24611...

Reviewers

The RB admin panel now includes settings for choosing between Whoosh
(the default) and Elasticsearch as search engines.

Set up Review Board to use Elasticsearch and was able to perform
searches after building the index.


Description From Last Updated

Let's swap these fields. The index depends on the URL. We should also increase the width of both fields. The …

chipx86chipx86

Can we make a constant on the class and then use that both here and in the SEARCH_ENGINE_CHOICES tuple?

daviddavid

'elasticsearch' imported but unused

reviewbotreviewbot

Since there's only one variable being formatted in, we can just use %s and positional-based formatting, which should shorten this …

daviddavid

How about "reviewboard"?

daviddavid

You can use the new constant(s) here, too.

daviddavid

You can use the new constant(s) here, too.

daviddavid

Add a blank line between these?

daviddavid

Hmm, this is a little weird. I think I'd prefer just attaching the change handler and then calling updateCacheFormDisplay() at …

daviddavid

This should fit on one line.

daviddavid

Same deal here.

daviddavid

'elasticsearch' imported but unused

reviewbotreviewbot

'elasticsearch' imported but unused

reviewbotreviewbot

Since these are POD constants, the variable names should be all caps. As it is where you used it I …

daviddavid

You could just make the jquery inline here rather than defining a variable and using add(): $('#row-search_results_per_page, #row-search_engine') .toggle(searchEnabled);

daviddavid

You could just make the jquery inline here rather than defining a variable.

daviddavid

You could just make the jquery inline here rather than defining a variable and using add(): $('#row-elasticsearch_index_name, #row-elasticsearch_url') .toggle(searchEnabled && …

daviddavid

There's an extra line here.

daviddavid

Something terrible has happened here with your editor. Please reindent. You can also combine the selectors into a single string …

daviddavid

Let's rename during import to search_engines, so it's clear and won't conflict in the future.

chipx86chipx86

I think something went horribly wrong with search/replace. We're now shouting WOOSH and ELASTICSEARCH in a lot of places that …

daviddavid

Missing docstring.

chipx86chipx86

'elasticsearch' imported but unused

reviewbotreviewbot

Swap these.

chipx86chipx86

Let's rename here as well.

chipx86chipx86

Missing a file docstring.

chipx86chipx86

Can we document both of these?

chipx86chipx86

Why the "WHOOSH"?

chipx86chipx86

Can be: $('#id_search_engine, #id_search_enable') .change(updateSearchFormDisplay);

daviddavid

? This isn't how it's listed in pypi.

chipx86chipx86

'elasticsearch' imported but unused

reviewbotreviewbot
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
  2. reviewboard/admin/forms.py (Diff revision 1)
     
     
     'elasticsearch' imported but unused
    
  3. 
      
brennie
brennie
david
  1. 
      
  2. reviewboard/admin/forms.py (Diff revision 1)
     
     

    Can we make a constant on the class and then use that both here and in the SEARCH_ENGINE_CHOICES tuple?

  3. reviewboard/admin/forms.py (Diff revision 1)
     
     
     
     
     

    Since there's only one variable being formatted in, we can just use %s and positional-based formatting, which should shorten this code a bit.

  4. reviewboard/admin/siteconfig.py (Diff revision 1)
     
     

    How about "reviewboard"?

  5. reviewboard/admin/siteconfig.py (Diff revision 1)
     
     

    You can use the new constant(s) here, too.

  6. reviewboard/admin/siteconfig.py (Diff revision 1)
     
     

    You can use the new constant(s) here, too.

  7. Add a blank line between these?

  8. Hmm, this is a little weird. I think I'd prefer just attaching the change handler and then calling updateCacheFormDisplay() at the end of the function.

  9. This should fit on one line.

  10. Same deal here.

  11. 
      
brennie
brennie
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: Pyflakes
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        reviewboard/search/engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        reviewboard/search/engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
  2. reviewboard/admin/forms.py (Diff revision 2)
     
     
     'elasticsearch' imported but unused
    
  3. reviewboard/admin/forms.py (Diff revision 3)
     
     
     'elasticsearch' imported but unused
    
  4. 
      
david
  1. 
      
  2. reviewboard/search/engines.py (Diff revision 3)
     
     
     
     

    Since these are POD constants, the variable names should be all caps. As it is where you used it I mistook it for a class name.

    Probably also don't need the blank line between them.

  3. You could just make the jquery inline here rather than defining a variable and using add():

    $('#row-search_results_per_page, #row-search_engine')
        .toggle(searchEnabled);
    
  4. You could just make the jquery inline here rather than defining a variable.

  5. You could just make the jquery inline here rather than defining a variable and using add():

    $('#row-elasticsearch_index_name, #row-elasticsearch_url')
        .toggle(searchEnabled && searchEngine === 'elasticsearch');
    
  6. There's an extra line here.

  7. reviewboard/templates/admin/general_settings.html (Diff revision 3)
     
     
     
     
     
     
     
     

    Something terrible has happened here with your editor. Please reindent.

    You can also combine the selectors into a single string with a comma.

  8. 
      
brennie
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        setup.py
        reviewboard/search/engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        setup.py
        reviewboard/search/engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
  2. reviewboard/admin/forms.py (Diff revision 4)
     
     
     'elasticsearch' imported but unused
    
  3. 
      
david
  1. 
      
  2. reviewboard/admin/forms.py (Diff revision 4)
     
     
     

    I think something went horribly wrong with search/replace. We're now shouting WOOSH and ELASTICSEARCH in a lot of places that we shouldn't be. Only the variable names should be all-caps.

  3. Can be:

    $('#id_search_engine, #id_search_enable')
        .change(updateSearchFormDisplay);
    
  4. 
      
chipx86
  1. I think rather than hard-coding everything for Whoosh and Elasticsearch, we should do what we do for auth, hosting services, etc. and have a concept of backends that are registered as entrypoints, with config forms. It's a little more work up-front, but it makes it a lot easier down the road for us or third-parties to support additional search engines (say, AWS's search services).

    We'd still go through Haystack for all this, so any extension providing such things would need to deal with that, but we'd have our side covered when it comes to settings and configuration.

  2. Let's swap these fields. The index depends on the URL.

    We should also increase the width of both fields. The URL especially.

  3. reviewboard/admin/forms.py (Diff revision 4)
     
     

    Let's rename during import to search_engines, so it's clear and won't conflict in the future.

  4. reviewboard/admin/forms.py (Diff revision 4)
     
     

    Missing docstring.

  5. reviewboard/admin/siteconfig.py (Diff revision 4)
     
     
     

    Swap these.

  6. reviewboard/admin/siteconfig.py (Diff revision 4)
     
     

    Let's rename here as well.

  7. reviewboard/search/engines.py (Diff revision 4)
     
     

    Missing a file docstring.

  8. reviewboard/search/engines.py (Diff revision 4)
     
     
     

    Can we document both of these?

  9. reviewboard/search/engines.py (Diff revision 4)
     
     

    Why the "WHOOSH"?

  10. setup.py (Diff revision 4)
     
     

    ? This isn't how it's listed in pypi.

    1. Find and replace gone wrong.

  11. 
      
brennie
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        reviewboard/search/search_engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        reviewboard/admin/siteconfig.py
        reviewboard/admin/forms.py
        reviewboard/search/search_engines.py
    
    Ignored Files:
        reviewboard/templates/admin/general_settings.html
    
    
  2. reviewboard/admin/forms.py (Diff revision 5)
     
     
     'elasticsearch' imported but unused
    
  3. 
      
david
  1. This looks good to me as-is, but I agree with Christian's comment about making this more pluggable. Perhaps in a follow-up change?

  2. 
      
brennie
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-3.0.x (feb91a6)
Loading...