Add a method for returning reviews that are accessible by a given user to ReviewManager
Review Request #12302 — Created May 25, 2022 and submitted
Currently there is no efficient way to query for reviews that are
accessible by a given user when usingReviews.objects
. This change adds a
method toReviewManager
such thatReviews.objects.accessible(user)
returns a
queryset of reviews that are accessible by the given user.
- Created
test_review_manager.py
file and moved in theReviewManager
tests fromtest_review.py
. - Created tests for
Review.objects.accessible
and ran all tests in
test_review_manager.py
with success.
Summary | ID | Author |
---|---|---|
932f8f52bcbe23fa2e680cd5af257f04ec4bcb00 | Michelle |
Description | From | Last Updated |
---|---|---|
F401 'django.contrib.auth.models.AnonymousUser' imported but unused |
reviewbot | |
F401 'djblets.testing.decorators.add_fixtures' imported but unused |
reviewbot | |
F401 'reviewboard.reviews.errors.RevokeShipItError' imported but unused |
reviewbot | |
F401 'reviewboard.reviews.models.Review' imported but unused |
reviewbot | |
F401 'reviewboard.reviews.models.ReviewRequest' imported but unused |
reviewbot | |
F401 'reviewboard.reviews.signals.review_ship_it_revoked' imported but unused |
reviewbot | |
F401 'reviewboard.reviews.signals.review_ship_it_revoking' imported but unused |
reviewbot | |
what do you mean by "should be consolidated?" perhaps I can address this TODO in my changes |
maubin | |
i feel iffy about changing the logic of the _query() function to this extent. now if we're querying for unpublished … |
maubin | |
i'm wondering if there's a way to add users to a local site |
maubin | |
Let's add a period at the end of this. |
david | |
E402 module level import not at top of file |
reviewbot | |
We can get rid of this now, in favor of LocalSite.ALL (which we'll want to document as a valid type … |
chipx86 | |
Since this method is new, we can default this to off. It's a bit inconsistent with other changes, but is … |
chipx86 | |
I think we should just require a User. Something I want to phase out is the whole user-or-username aspect of … |
chipx86 | |
When we have functions with multiple keyword arguments, it's best to do one per line. That helps with both readability … |
chipx86 | |
These queries are very expensive, full of nested joins and stuff. We don't want to blindly OR those together, as … |
chipx86 | |
We can just do an assert for str inside the else, to elimate an isinstance check. |
chipx86 | |
The more fixtures we load, the more expensive per-test setup. We should avoid test_site except for tests that specifically need … |
chipx86 | |
Two important things here: Private functions always go after public functions. This function is very specific to certain tests we … |
chipx86 | |
To keep code consistent, let's always use review_request and review as variable names. |
chipx86 | |
assertIn / assertNotIn can be very misleading. It doesn't always tell us the whole story. For instance, we might expect … |
chipx86 | |
continuation line over-indented for visual indent Column: 38 Error code: E127 |
reviewbot | |
continuation line over-indented for visual indent Column: 38 Error code: E127 |
reviewbot | |
No blank line after method-level docstrings. |
chipx86 | |
Let's change this to "Added the ..." That'll be more consistent, and reads a bit better. |
chipx86 |
- Change Summary:
-
Mostly done but I have a question about testing with local sites (see comment)
- Testing Done:
-
~ - Created
test_review_manager.py
file and moved in theReviewManager
tests fromtest_review.py
~ - Will add tests to test
Review.objects.accessible
~ - Created
test_review_manager.py
file and moved in theReviewManager
tests fromtest_review.py
.
~ - Created tests for
Review.objects.accessible
and ran all tests in
test_review_manager.py
with success.
- Created
- Commits:
-
Summary ID Author 21577170115c559e3ab5d942b4f07cffc9cb3c91 Michelle 45a2f1d2bf036b6a644a5219f1b370664dd8de09 Michelle
Checks run (2 succeeded)
- Commits:
-
Summary ID Author 45a2f1d2bf036b6a644a5219f1b370664dd8de09 Michelle be98ccefff4eaf42467620dc0f7e8744c9f793f3 Michelle
Checks run (2 succeeded)
-
-
-
i feel iffy about changing the logic of the
_query()
function to this extent. now if we're querying for unpublished reviews and a user is given, it will only return unpublished reviews that are owned by that user. i made this change because that's what i need for theaccessible()
function to work properly, otherwise it would return drafts that are owned by other users. i tried to think of other ways foraccessible()
to only return drafts owned by the given user, but none worked out. if you see any ways to achieve this without having to modify_query()
, or if it's fine to modify_query()
like this let me know. -
- Change Summary:
-
Ready for review.
- Summary:
-
[WIP] Add a method for returning reviews that are accessible by a given user to ReviewManagerAdd a method for returning reviews that are accessible by a given user to ReviewManager
- Commits:
-
Summary ID Author be98ccefff4eaf42467620dc0f7e8744c9f793f3 Michelle 4b6a9d6c995a4cd6c8524d5ccf2ab4b0cae1adac Michelle
Checks run (2 succeeded)
- Change Summary:
-
cleaned up query code, added asserts for
user_or_username
's type - Commits:
-
Summary ID Author 4b6a9d6c995a4cd6c8524d5ccf2ab4b0cae1adac Michelle 974b7bb0f9741a07688a93305bbfc5905bec9034 Michelle
- Commits:
-
Summary ID Author 974b7bb0f9741a07688a93305bbfc5905bec9034 Michelle 0d172585b59f028897d1addcc0bde5c0e2b4846b Michelle
Checks run (2 succeeded)
-
-
We can get rid of this now, in favor of
LocalSite.ALL
(which we'll want to document as a valid type forlocal_site
-- I think I have this in one of my changes). -
Since this method is new, we can default this to off. It's a bit inconsistent with other changes, but is probably the correct way to go.
-
I think we should just require a
User
. Something I want to phase out is the whole user-or-username aspect of the other methods, because it eventually leads to unwanted extra queries.The reason is that, for the kinds of checks we perform, we need a
User
anyway. So the query function has to get one. However, if the code calling this eventually needs the sameUser
, they have to query for it again, which is therefore slower.Requiring a
User
ensures the caller has to think about when it's getting a user and how. -
When we have functions with multiple keyword arguments, it's best to do one per line. That helps with both readability and maintainability.
-
These queries are very expensive, full of nested joins and stuff. We don't want to blindly OR those together, as it'll get pretty hairy pretty quickly.
Let's look at how this looks. We'll go with a simple example. Just one
_query()
:str(Review.objects._query(some_non_admin_user, public=True, status=None, filter_private=True).query)
Which gives us something like the following (after I run it through a formatter):
SELECT DISTINCT "reviews_review"."id", "reviews_review"."review_request_id", "reviews_review"."user_id", "reviews_review"."timestamp", "reviews_review"."public", "reviews_review"."ship_it", "reviews_review"."base_reply_to_id", "reviews_review"."email_message_id", "reviews_review"."time_emailed", "reviews_review"."body_top", "reviews_review"."body_top_rich_text", "reviews_review"."body_bottom", "reviews_review"."body_bottom_rich_text", "reviews_review"."body_top_reply_to_id", "reviews_review"."body_bottom_reply_to_id", "reviews_review"."extra_data", "reviews_review"."rich_text", "reviews_review"."reviewed_diffset_id" FROM "reviews_review" INNER JOIN "reviews_reviewrequest" ON ( "reviews_review"."review_request_id" = "reviews_reviewrequest"."id" ) LEFT OUTER JOIN "reviews_reviewrequest_target_people" ON ( "reviews_reviewrequest"."id" = "reviews_reviewrequest_target_people"."reviewrequest_id" ) LEFT OUTER JOIN "reviews_reviewrequest_target_groups" ON ( "reviews_reviewrequest"."id" = "reviews_reviewrequest_target_groups"."reviewrequest_id" ) WHERE ( "reviews_review"."public" = TRUE AND "reviews_review"."base_reply_to_id" IS NULL AND "reviews_reviewrequest"."local_site_id" IS NULL AND ( "reviews_review"."user_id" = 1 OR ( ( "reviews_reviewrequest"."repository_id" IS NULL OR "reviews_reviewrequest"."repository_id" IN (1, 3, 4,) ) AND ( "reviews_reviewrequest_target_people"."user_id" = 1 OR "reviews_reviewrequest_target_groups"."group_id" IS NULL OR "reviews_reviewrequest_target_groups"."group_id" IN (1, 2, 3, 4) ) ) ) ) ORDER BY "reviews_review"."timestamp" ASC
If we run another (with
public=False
), we get something similar.Now let's do the OR:
str((Review.objects._query(non_admin_user, public=True, status=None, filter_private=True) | Review.objects._query(non_admin_user, public=False, status=None, filter_private=True)).query)
We get:
SELECT DISTINCT "reviews_review"."id", "reviews_review"."review_request_id", "reviews_review"."user_id", "reviews_review"."timestamp", "reviews_review"."public", "reviews_review"."ship_it", "reviews_review"."base_reply_to_id", "reviews_review"."email_message_id", "reviews_review"."time_emailed", "reviews_review"."body_top", "reviews_review"."body_top_rich_text", "reviews_review"."body_bottom", "reviews_review"."body_bottom_rich_text", "reviews_review"."body_top_reply_to_id", "reviews_review"."body_bottom_reply_to_id", "reviews_review"."extra_data", "reviews_review"."rich_text", "reviews_review"."reviewed_diffset_id" FROM "reviews_review" INNER JOIN "reviews_reviewrequest" ON ( "reviews_review"."review_request_id" = "reviews_reviewrequest"."id" ) LEFT OUTER JOIN "reviews_reviewrequest_target_people" ON ( "reviews_reviewrequest"."id" = "reviews_reviewrequest_target_people"."reviewrequest_id" ) LEFT OUTER JOIN "reviews_reviewrequest_target_groups" ON ( "reviews_reviewrequest"."id" = "reviews_reviewrequest_target_groups"."reviewrequest_id" ) WHERE ( ( "reviews_review"."public" = TRUE AND "reviews_review"."base_reply_to_id" IS NULL AND "reviews_reviewrequest"."local_site_id" IS NULL AND ( "reviews_review"."user_id" = 1 OR ( ( "reviews_reviewrequest"."repository_id" IS NULL OR "reviews_reviewrequest"."repository_id" IN (1, 2, 3, 4) ) AND ( "reviews_reviewrequest_target_people"."user_id" = 1 OR "reviews_reviewrequest_target_groups"."group_id" IS NULL OR "reviews_reviewrequest_target_groups"."group_id" IN (1, 2, 3, 4) ) ) ) ) OR ( "reviews_review"."public" = FALSE AND "reviews_review"."base_reply_to_id" IS NULL AND "reviews_reviewrequest"."local_site_id" IS NULL AND ( "reviews_review"."user_id" = 1 OR ( ( "reviews_reviewrequest"."repository_id" IS NULL OR "reviews_reviewrequest"."repository_id" IN (1, 2, 3 3, 4) ) AND ( "reviews_reviewrequest_target_people"."user_id" = 1 OR "reviews_reviewrequest_target_groups"."group_id" IS NULL OR "reviews_reviewrequest_target_groups"."group_id" IN (1, 2, 3, 4) ) ) ) ) ) ORDER BY "reviews_review"."timestamp" ASC
Look how expensive that just go with the
OR
. Way more complicated, and the database is going to take who knows what strategy to check things -- probably a row scan, meaning it searches nearly every single row in the database. Sort of a worst-case scenario. So let's help with that.Since
public=True | public=False
really just means "any value forpublic
, we really just want to eliminate that part of the query. And then have one block of ACL checks.We can do this by allowing
public
to beNone
, and just omitting theQ(public=...)
if it isNone
. That gets us everything we need, keeping the simpler, faster (still expensive but soon less so) version of the query.Database query work tends to be more complex than it appears on the surface, especially since the Django ORM hides a lot of complexity from us. There isn't a great way to test complexity of queries in Django, so it boils down to running the queries like above and seeing what we get (and possibly using SQL like
EXPLAIN
to figure out what the database would do with it). Happy to go into those tricks offline. -
-
The more fixtures we load, the more expensive per-test setup. We should avoid
test_site
except for tests that specifically need it, and same withtest_scmtools
. -
Two important things here:
- Private functions always go after public functions.
- This function is very specific to certain tests we have now. There are more coming, where these won't be appropriate, and that can lead to shoe-horning in additional behaviors (same problem our existing fixtures have).
I suggest just passing in
AnonymousUser()
where we need it, and looking up the users where we need them. -
-
assertIn
/assertNotIn
can be very misleading. It doesn't always tell us the whole story. For instance, we might expect that because an item is or is not in the results that we've gotten the results we expect, but what if we get a review request we don't expect? Or assumed there'd be one in there where there isn't?Not necessarily an issue for all of these tests, but since we're testing for accessibility, we want to be very explicit. Let's instead do:
self.assertQuerysetEqual( Review.objects.accessible(...), [list of review request options])
Same applies to all tests.
- Change Summary:
-
- Got rid of
show_all_local_sites
inReviewManager
- Defaults to not returning distinct results in
ReviewManager.accessible
- Uses
user
parameter instead ofuser_or_username
inReviewManager
- Fleshed out the docs for all functions in
ReviewManager
- Coding style fixes
- Switch to using
assertQuerysetEqual
in tests - Applies fixtures to specific tests instead of all
- Removed
_create_users()
in tests
- Got rid of
- Commits:
-
Summary ID Author 0d172585b59f028897d1addcc0bde5c0e2b4846b Michelle 3286eb541c75d01c997d8fed9a0ee9394363b298 Michelle
Checks run (2 succeeded)
- Change Summary:
-
Changed a copy paste mistake of referring to "comments" instead of "reviews"
- Commits:
-
Summary ID Author 3286eb541c75d01c997d8fed9a0ee9394363b298 Michelle 5efcd2ac765cc2f48e60823037042e4eca7ba139 Michelle
Checks run (2 succeeded)
- Change Summary:
-
Changed another mention of "comments" and added another backtick to wrap None in a docstring
- Commits:
-
Summary ID Author 5efcd2ac765cc2f48e60823037042e4eca7ba139 Michelle 3c723a5d88344567e29e790391daa4e3cb22033d Michelle
Checks run (2 succeeded)
- Change Summary:
-
- Moved access control logic for removing drafts that do not belong to the user into the
_query
method - Removed unnecessary
transform
parameter fromassertQuerysetEqual
- Moved access control logic for removing drafts that do not belong to the user into the
- Commits:
-
Summary ID Author 3c723a5d88344567e29e790391daa4e3cb22033d Michelle 77093de89e972b8029468b7725402c5b6f3de496 Michelle
- Change Summary:
-
fix flake8 issues
- Commits:
-
Summary ID Author 77093de89e972b8029468b7725402c5b6f3de496 Michelle 474c42199fda5a34dab821559fa056f1cc1e0a95 Michelle
Checks run (2 succeeded)
- Change Summary:
-
- Added
assertNumQueries
to tests
- Added
- Commits:
-
Summary ID Author 474c42199fda5a34dab821559fa056f1cc1e0a95 Michelle ca4a01590d7e9b12dba20d832970517a9aecfd22 Michelle
Checks run (2 succeeded)
- Change Summary:
-
Takes advantage of the cache to reduce the number of queries in one test
- Commits:
-
Summary ID Author ca4a01590d7e9b12dba20d832970517a9aecfd22 Michelle 4603b13bce0d0847ada8e344a2fb243a9bc9e356 Michelle
Checks run (2 succeeded)
- Change Summary:
-
Added more extensive test coverage
- Commits:
-
Summary ID Author 4603b13bce0d0847ada8e344a2fb243a9bc9e356 Michelle b59874bf9d8f7062cb072c9d8e1514097a75b51e Michelle
Checks run (2 succeeded)
- Change Summary:
-
Add a missing docstring and version addeds
- Commits:
-
Summary ID Author b59874bf9d8f7062cb072c9d8e1514097a75b51e Michelle 93750baa5155c41694610d9f781a85fc3d9b5e85 Michelle
Checks run (2 succeeded)
- Change Summary:
-
- Removed a blank line between a method docstring and the method body
- Changed to "Added the"
- Commits:
-
Summary ID Author 93750baa5155c41694610d9f781a85fc3d9b5e85 Michelle 20a345390429c346b00c0df85c3ae1b709a19a1a Michelle - Diff:
-
Revision 16 (+1317 -2845)