Optimize data fetching in datagrids and allow for augmented querysets.

Review Request #1338 — Created Jan. 4, 2010 and submitted




Optimize data fetching in datagrids and allow for augmented querysets.

This change makes it possible to have more efficient datagrids. The default
Column.render_data function now stores a cache of models spanning all rows,
keyed off by the models' primary keys. What this does is prevent, say,
a column displaying 'obj.user.username' from querying the database for the
user more than once per user instance. Previously, if a datagrid showed 20
rows of data with 10 unique usernames, it would still query the database 20
times for the users. Now it would only query 10 times.

The second change is to provide a new Column.augment_query function. While
processing the queryset and after providing any datagrid-wide customizations,
it will go through each column used in that datagrid and allow the columns
to query for additional data. Columns can return a new queryset that, for
example, performs an inner SELECT on some related table to cache the value,
which will only be done on the initial query instead of needing to be queried
Tested these with my new changes to Review Board.

On a dashboard with 20 review requests and only 3 unique users, we only had 3 fetches for users, instead of 20.

The Review Board change also makes extensive use of augment_query, which drastically reduced the queries made.