• 
      

    Rework our Haystack search support to use a forwarding backend approach.

    Review Request #11348 — Created Dec. 23, 2020 and submitted — Latest diff uploaded

    Information

    Review Board
    release-4.0.x

    Reviewers

    Haystack's never been good at dealing with changes to the configured
    search backend, but more recent versions have become more difficult,
    with more and more objects being initialized and stored on import. This
    means that our prior attempts to clear caches when changing backends
    just wasn't practical anymore.

    This change implements the same approach we use for cache backends. We
    now have a single main backend we assign to Haystack, which then
    forwards on requests to the configured backend. This gives us a lot more
    control and is more future-proof.

    The one thing that can't be addressed here is a change they made to
    SearchView. They construct a SearchQuerySet at class definition
    time, which then pulls the BaseSearchQuery from the backend and stores
    it. The only workaround for this is pretty straight-forward: we
    construct a new SearchQuerySet at HTTP dispatch time.

    With this comes a few additional fixes, primarily in our unit tests. We
    sometimes hit random breakages in search tests, due to Whoosh being used
    in asynchronous mode when writing to the disk. This sometimes resulted
    in newly-indexed data failing to be read ("fixed" by sleeping for .1
    second) and sometimes resulted in a prior test clobbering a successive
    one (fixed by giving each run its own index directory within a temp
    directory).

    There's also a couple packaging updates. We now depend on the latest
    Haystack series prior to 3.0 (which is Python 3-only), and we depend on
    elasticsearch (which is a pretty small module with a very specific
    version range we require).

    All unit tests pass (except some unrelated breakages I'm sorting outs
    separately).

    Ran the suite several times and didn't hit any more random breakages,
    though I'm not ruling out the possibility.

    Tested searching on both Whoosh and Elasticsearch. Switched back-and-forth
    between an Elasticsearch configuration with a populated index and a Whoosh
    configuration with an empty index. I was able to dynamically switch and use
    the proper engine without restarting the web server. Confirmed the right
    backend objects were in use via print statements.

    Commits

    Files