Perform incremental search reindexing of users based on the last_login field.

Review Request #10645 — Created July 24, 2019 and submitted

chipx86
Review Board
release-3.0.x
ad73ae9...
reviewboard

When performing an incremental reindexing of users, Haystack emits
warnings that it has no way of determining the age of an entry for
users, since there's no sort of "last updated" field available.

We do have a workaround, in that Review Board does update the
last_login field when users access the site, rather than just when
they explicitly log in. We can tie updates to that.

What this basically means is that we will re-index any active accounts
(which might have information changed), but if information is changed
with an older account, it won't be noticed without performing a full
re-index.

Performing an incremental reindexing and saw that the warnings went
away, and the new information on a user was indexed.

Description From Last Updated

Hmm. This potentially means reindexing every user every 30 minutes. That seems excessive.

daviddavid
david
  1. 
      
  2. Hmm. This potentially means reindexing every user every 30 minutes. That seems excessive.

    1. Without it, it reindexes every user :/ Basically omits a __lte= and __gte= query.

    2. updated_field = self.get_updated_field()
      
      update_field_msg = ("No updated date field found for '%s' "
                          "- not restricting by age.") % model.__name__
      
      if start_date:
          if updated_field:
              extra_lookup_kwargs['%s__gte' % updated_field] = start_date
          else:
              warnings.warn(update_field_msg)
      
      if end_date:
          if updated_field:
              extra_lookup_kwargs['%s__lte' % updated_field] = end_date
          else:
              warnings.warn(update_field_msg)
      
      ...
      
      return index_qs.filter(**extra_lookup_kwargs).order_by(model._meta.pk.name)
      
    3. OK then.

  3. 
      
david
  1. Ship It!
  2. 
      
chipx86
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-3.0.x (e394b02)
Loading...