• 
      

    Add support for Elasticsearch

    Review Request #8209 — Created June 2, 2016 and submitted

    Information

    Review Board
    release-3.0.x
    2a24611...

    Reviewers

    The RB admin panel now includes settings for choosing between Whoosh
    (the default) and Elasticsearch as search engines.

    Set up Review Board to use Elasticsearch and was able to perform
    searches after building the index.


    Description From Last Updated

    Let's swap these fields. The index depends on the URL. We should also increase the width of both fields. The …

    chipx86 chipx86

    Can we make a constant on the class and then use that both here and in the SEARCH_ENGINE_CHOICES tuple?

    david david

    'elasticsearch' imported but unused

    reviewbot reviewbot

    Since there's only one variable being formatted in, we can just use %s and positional-based formatting, which should shorten this …

    david david

    How about "reviewboard"?

    david david

    You can use the new constant(s) here, too.

    david david

    You can use the new constant(s) here, too.

    david david

    Add a blank line between these?

    david david

    Hmm, this is a little weird. I think I'd prefer just attaching the change handler and then calling updateCacheFormDisplay() at …

    david david

    This should fit on one line.

    david david

    Same deal here.

    david david

    'elasticsearch' imported but unused

    reviewbot reviewbot

    'elasticsearch' imported but unused

    reviewbot reviewbot

    Since these are POD constants, the variable names should be all caps. As it is where you used it I …

    david david

    You could just make the jquery inline here rather than defining a variable and using add(): $('#row-search_results_per_page, #row-search_engine') .toggle(searchEnabled);

    david david

    You could just make the jquery inline here rather than defining a variable.

    david david

    You could just make the jquery inline here rather than defining a variable and using add(): $('#row-elasticsearch_index_name, #row-elasticsearch_url') .toggle(searchEnabled && …

    david david

    There's an extra line here.

    david david

    Something terrible has happened here with your editor. Please reindent. You can also combine the selectors into a single string …

    david david

    Let's rename during import to search_engines, so it's clear and won't conflict in the future.

    chipx86 chipx86

    I think something went horribly wrong with search/replace. We're now shouting WOOSH and ELASTICSEARCH in a lot of places that …

    david david

    Missing docstring.

    chipx86 chipx86

    'elasticsearch' imported but unused

    reviewbot reviewbot

    Swap these.

    chipx86 chipx86

    Let's rename here as well.

    chipx86 chipx86

    Missing a file docstring.

    chipx86 chipx86

    Can we document both of these?

    chipx86 chipx86

    Why the "WHOOSH"?

    chipx86 chipx86

    Can be: $('#id_search_engine, #id_search_enable') .change(updateSearchFormDisplay);

    david david

    ? This isn't how it's listed in pypi.

    chipx86 chipx86

    'elasticsearch' imported but unused

    reviewbot reviewbot
    reviewbot
    1. Tool: Pyflakes
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
    2. reviewboard/admin/forms.py (Diff revision 1)
       
       
      Show all issues
       'elasticsearch' imported but unused
      
    3. 
        
    brennie
    brennie
    david
    1. 
        
    2. reviewboard/admin/forms.py (Diff revision 1)
       
       
      Show all issues

      Can we make a constant on the class and then use that both here and in the SEARCH_ENGINE_CHOICES tuple?

    3. reviewboard/admin/forms.py (Diff revision 1)
       
       
       
       
       
      Show all issues

      Since there's only one variable being formatted in, we can just use %s and positional-based formatting, which should shorten this code a bit.

    4. reviewboard/admin/siteconfig.py (Diff revision 1)
       
       
      Show all issues

      How about "reviewboard"?

    5. reviewboard/admin/siteconfig.py (Diff revision 1)
       
       
      Show all issues

      You can use the new constant(s) here, too.

    6. reviewboard/admin/siteconfig.py (Diff revision 1)
       
       
      Show all issues

      You can use the new constant(s) here, too.

    7. Show all issues

      Add a blank line between these?

    8. Show all issues

      Hmm, this is a little weird. I think I'd prefer just attaching the change handler and then calling updateCacheFormDisplay() at the end of the function.

    9. Show all issues

      This should fit on one line.

    10. Show all issues

      Same deal here.

    11. 
        
    brennie
    brennie
    reviewbot
    1. Tool: Pyflakes
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: Pyflakes
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          reviewboard/search/engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          reviewboard/search/engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
    2. reviewboard/admin/forms.py (Diff revision 2)
       
       
      Show all issues
       'elasticsearch' imported but unused
      
    3. reviewboard/admin/forms.py (Diff revision 3)
       
       
      Show all issues
       'elasticsearch' imported but unused
      
    4. 
        
    david
    1. 
        
    2. reviewboard/search/engines.py (Diff revision 3)
       
       
       
       
      Show all issues

      Since these are POD constants, the variable names should be all caps. As it is where you used it I mistook it for a class name.

      Probably also don't need the blank line between them.

    3. Show all issues

      You could just make the jquery inline here rather than defining a variable and using add():

      $('#row-search_results_per_page, #row-search_engine')
          .toggle(searchEnabled);
      
    4. Show all issues

      You could just make the jquery inline here rather than defining a variable.

    5. Show all issues

      You could just make the jquery inline here rather than defining a variable and using add():

      $('#row-elasticsearch_index_name, #row-elasticsearch_url')
          .toggle(searchEnabled && searchEngine === 'elasticsearch');
      
    6. Show all issues

      There's an extra line here.

    7. reviewboard/templates/admin/general_settings.html (Diff revision 3)
       
       
       
       
       
       
       
       
      Show all issues

      Something terrible has happened here with your editor. Please reindent.

      You can also combine the selectors into a single string with a comma.

    8. 
        
    brennie
    reviewbot
    1. Tool: Pyflakes
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          setup.py
          reviewboard/search/engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          setup.py
          reviewboard/search/engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
    2. reviewboard/admin/forms.py (Diff revision 4)
       
       
      Show all issues
       'elasticsearch' imported but unused
      
    3. 
        
    david
    1. 
        
    2. reviewboard/admin/forms.py (Diff revision 4)
       
       
       
      Show all issues

      I think something went horribly wrong with search/replace. We're now shouting WOOSH and ELASTICSEARCH in a lot of places that we shouldn't be. Only the variable names should be all-caps.

    3. Show all issues

      Can be:

      $('#id_search_engine, #id_search_enable')
          .change(updateSearchFormDisplay);
      
    4. 
        
    chipx86
    1. I think rather than hard-coding everything for Whoosh and Elasticsearch, we should do what we do for auth, hosting services, etc. and have a concept of backends that are registered as entrypoints, with config forms. It's a little more work up-front, but it makes it a lot easier down the road for us or third-parties to support additional search engines (say, AWS's search services).

      We'd still go through Haystack for all this, so any extension providing such things would need to deal with that, but we'd have our side covered when it comes to settings and configuration.

    2. Show all issues

      Let's swap these fields. The index depends on the URL.

      We should also increase the width of both fields. The URL especially.

    3. reviewboard/admin/forms.py (Diff revision 4)
       
       
      Show all issues

      Let's rename during import to search_engines, so it's clear and won't conflict in the future.

    4. reviewboard/admin/forms.py (Diff revision 4)
       
       
      Show all issues

      Missing docstring.

    5. reviewboard/admin/siteconfig.py (Diff revision 4)
       
       
       
      Show all issues

      Swap these.

    6. reviewboard/admin/siteconfig.py (Diff revision 4)
       
       
      Show all issues

      Let's rename here as well.

    7. reviewboard/search/engines.py (Diff revision 4)
       
       
      Show all issues

      Missing a file docstring.

    8. reviewboard/search/engines.py (Diff revision 4)
       
       
       
      Show all issues

      Can we document both of these?

    9. reviewboard/search/engines.py (Diff revision 4)
       
       
      Show all issues

      Why the "WHOOSH"?

    10. setup.py (Diff revision 4)
       
       
      Show all issues

      ? This isn't how it's listed in pypi.

      1. Find and replace gone wrong.

    11. 
        
    brennie
    reviewbot
    1. Tool: Pyflakes
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          reviewboard/search/search_engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
      
      Tool: PEP8 Style Checker
      Processed Files:
          reviewboard/admin/siteconfig.py
          reviewboard/admin/forms.py
          reviewboard/search/search_engines.py
      
      Ignored Files:
          reviewboard/templates/admin/general_settings.html
      
      
    2. reviewboard/admin/forms.py (Diff revision 5)
       
       
      Show all issues
       'elasticsearch' imported but unused
      
    3. 
        
    david
    1. This looks good to me as-is, but I agree with Christian's comment about making this more pluggable. Perhaps in a follow-up change?

    2. 
        
    brennie
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-3.0.x (feb91a6)