Add a specialization of CounterField for tracking model relations.

Review Request #6248 — Created Aug. 20, 2014 and submitted

Information

Djblets
release-0.8.x
a916a2d...

Reviewers

RelationCounterField tracks the counts on one end of a ForeignKey or
ManyToManyField relation, providing a convenient and efficient way of
operating based on how many objects are related to the model.

This takes a single parameter, which is the relation name. That can be
the field name of a local ForeignKey/ManyToManyField, or it can be
the related_name on the model that's provided by another model's
ForeignKey/ManyToManyField. For example:

reviews_count = RelationCounterField('reviews')

When there are Django-initiated state changes on the opposite end of
the relation, such as newly added or deleted models, the counters will
update with the new count.

These counts should be accurate so long as the operations are performed
on model instances. Raw SQL queries will cause the models to get out of
sync. When this happens, the counts can be recomputed by issuing a
reinit_<fieldname> call.

RelationCounterField itself doesn't actually do much, aside from set
things up. All the magic happens in InstanceState and RelationTracker.

InstanceState holds information on a loaded model instance/relation name
pair, and all RelationCounterFields that follow that pair. It has
functions for operation on all the fields on a model following that
relation. (Normally there will be 1, but the design works out better to
just allow for more than one, since otherwise we actually have to track
more things to deal with conflicting state.)

RelationTracker is the real heart of RelationCounterField. There's one
RelationTracker for every model class/relation name pair. It figures out
information on the relation, listens to signals, and makes the
appropriate database and instance updates (using InstanceState).

Since Django's signals and API for all this are a little less useful
than I'd like, there's a lot of calculations that have to go on and some
state tracking we have to do. I've tried to document how it all works,
but there's likely room for improvement.

Unit tests were written to handle every case I could throw at it, along
with ensuring all query counts are consistent with what I'd expect.

Wrote a lot of unit tests. They all pass.

For the query counts in the unit tests, I added up all the queries for
the operation in both Django's code paths and ours. I then saved those as
constants and referenced them everywhere they made sense. The unit tests
ended up matching those query counts.

Description From Last Updated

Col: 80 E501 line too long (80 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (82 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (83 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (80 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (83 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (80 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (81 > 79 characters)

reviewbotreviewbot

Col: 80 E501 line too long (83 > 79 characters)

reviewbotreviewbot

Col: 25 E127 continuation line over-indented for visual indent

reviewbotreviewbot

Col: 5 E303 too many blank lines (2)

reviewbotreviewbot

Col: 9 E265 block comment should start with '# '

reviewbotreviewbot

Col: 5 E303 too many blank lines (2)

reviewbotreviewbot

Col: 5 E303 too many blank lines (2)

reviewbotreviewbot

Col: 5 E303 too many blank lines (2)

reviewbotreviewbot

local variable 'rel_model' is assigned to but never used

reviewbotreviewbot

Leftover debug code?

daviddavid

Remove this line?

daviddavid

Col: 25 E127 continuation line over-indented for visual indent

reviewbotreviewbot

Col: 25 E127 continuation line over-indented for visual indent

reviewbotreviewbot
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
  2. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (80 > 79 characters)
    
  3. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (82 > 79 characters)
    
  4. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (83 > 79 characters)
    
  5. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (80 > 79 characters)
    
  6. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (83 > 79 characters)
    
  7. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (80 > 79 characters)
    
  8. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (81 > 79 characters)
    
  9. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 80
     E501 line too long (83 > 79 characters)
    
  10. djblets/db/fields.py (Diff revision 1)
     
     
    Show all issues
    Col: 25
     E127 continuation line over-indented for visual indent
    
  11. Show all issues
    Col: 5
     E303 too many blank lines (2)
    
  12. Show all issues
    Col: 9
     E265 block comment should start with '# '
    
  13. Show all issues
    Col: 5
     E303 too many blank lines (2)
    
  14. Show all issues
    Col: 5
     E303 too many blank lines (2)
    
  15. Show all issues
    Col: 5
     E303 too many blank lines (2)
    
  16. Show all issues
     local variable 'rel_model' is assigned to but never used
    
  17. 
      
chipx86
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
  2. djblets/db/fields.py (Diff revision 2)
     
     
    Show all issues
    Col: 25
     E127 continuation line over-indented for visual indent
    
  3. 
      
david
  1. I have to admit, it kind of scares me to be making any changes to this stuff.

    1. Me too, but it's pretty well-tested at least. A lot of this would be easier if Django just had some nicer signals...

  2. djblets/db/fields.py (Diff revision 2)
     
     
    Show all issues

    Leftover debug code?

  3. djblets/db/fields.py (Diff revision 2)
     
     
    Show all issues

    Remove this line?

  4. 
      
chipx86
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        djblets/db/tests/test_relation_counter_field.py
        djblets/db/fields.py
    
    
  2. djblets/db/fields.py (Diff revision 3)
     
     
    Show all issues
    Col: 25
     E127 continuation line over-indented for visual indent
    
  3. 
      
david
  1. Ship It!

  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-0.8.x (f79265c)