Significantly improve performance of the datagrid paginators.
Review Request #13374 — Created Oct. 23, 2023 and submitted
This updates the datagrid querysets to set
distinct=Falseand to omit
with_counts=True. This simplifies the base querysets for all the
datagrids, ensuring that the datagrid code can generate a slim and
efficient queryset for computing total item counts for pagination. Prior
to this, the querysets were in the form of:
SELECT COUNT(*) FROM (SELECT DISTINCT ...);
That was causing all results to be fetched and then for a total count to
be computed based off that. Certainly not efficient, and the source of
problems in the past for some users.
The data queryset still uses distinct, which is set by the datagrid code
itself. Although we can now turn this off, we are going to keep it on
for now, with plans to test removing the distinct flag for querysets in
Review Board 7.
Unit tests pass.
Been testing all the datagrids with this change, ensuring that the counts
seem fine and that there are no noticeable regressions.