Make accessibility checks more robust for Reviews and Comments queries
Review Request #12424 — Created June 29, 2022 and submitted
Recently, it was discovered that when querying for reviews there may be
duplicateQ(public=True)
's that appear in the queries to fetch the reviews.
The cause of this duplication lies inReviewManager
's_query
method, which
handles the logic for querying for reviews. If an unauthenticated or anonymous
user was querying for the reviews, thepublic
query field would be set twice,
once to thepublic
value that was passed and once toTrue
, with the latter
ensuring that only public reviews get returned to unauthenticated/anonymous
users. This however makes the query less efficient and can lead to the wrong
results being included in the returned queryset if aFalse
value forpublic
was passed in.To fix this, we ensure that the
public
query field is only set once, for any
type of user, inReviewManager._query
. This fix was applied to
CommentManager._query
as well.This change also adds a
public
parameter toReviewManager.accessible
to
ensure that thepublic
parameter inReviewManager._query
handles all of
the logic surrounding public and unpublished reviews. Previously, you would
have to set thepublic
query field using theextra_query
parameter in
ReviewManager.accessible
, which could also lead to duplicate or
conflictingQ(public=
queries.Finally, this change also improves the tests for the
ReviewManager
and
CommentManager
. The tests are more clear, organized and well documented,
they useassertQueries
, and have more comprehensive test coverage surrounding
access control.
- Added new unit tests for
ReviewManager.accessible
with setting the
public
parameter. - Ran all unit tests with success.
Summary | ID |
---|---|
321568393659002f0d6d41524cdf9e1a4e9c19cd |
Description | From | Last Updated |
---|---|---|
How should we check that the diffset related to the review is accessible by the user? I see the logic … |
maubin | |
I think we can have a more performant query if this becomes something like: q &= ( Q(user=user) | (Q(public=True) … |
chipx86 | |
Seems like a good performance improvement. Let's document these conditions for an empty result with a comment. |
chipx86 | |
So going through the other changes for some duplicate object fixes I'm putting together, I think we need to take … |
chipx86 | |
For booleans, we allow ('1', 'true', 'True'). Not ideal to repeat this everywhere, but let's call that a problem for … |
chipx86 | |
This should be review__public, right? If so, and no tests failed, this means we're missing a test. |
chipx86 | |
Imports should be in alphabetical order. |
chipx86 | |
Comments should generally end with a period. |
chipx86 | |
Blank line before new blocks (same below). |
chipx86 | |
Should I consolidate these common queries into a variable or something? Then use the variable instead of repeating the full … |
maubin | |
For multi-line lists, we generally do: [ review1, review2, ..., review5, ..., ] Or, when appropriate, one item per row … |
chipx86 | |
Given this is one Q, we can just put this on the same line. Same in other places. |
chipx86 | |
Another example of where we want periods at the end of the sentence. Really applies to all such comments. |
chipx86 | |
trailing whitespace Column: 38 Error code: W291 |
reviewbot | |
Missing a trailing comma (we always want to terminate multi-column list items with a trailing comma to minimize line changes … |
chipx86 |
-
-
I think we can have a more performant query if this becomes something like:
q &= ( Q(user=user) | (Q(public=True) & (repo_q & Q(review_request__target_people=user) | group_q)) )
That allows the query planner to have to consider only one set of rows for a
public=True
, and short-circuits that if we know the user matches.It does mean repeating some of that above, but we can just take the
(repo_q ...)
bits and put that into a reusableacl_check_q
or something. -
Seems like a good performance improvement. Let's document these conditions for an empty result with a comment.
-
So going through the other changes for some duplicate object fixes I'm putting together, I think we need to take the opportunity to better organize these tests so it's explicitly clear what data we're working with.
As such, with these tests (and ideally we'd update the others to match, but let's start here), let's have a pattern of putting a comment above each distinct group of related data explaining what access policy that's testing. For example:
# Published review on a publicly-accessible review request. review1 = self.create_review(review_request, publish=True) ... # Draft review on a publicly-accessible review request. ... # Published review on a private repository the user has access to. ... # Published review on a private repository the user does not have access to. ...
etc.
That way it's very easy for us to see what data we're working with, and it's very easy to know where the code should live.
Bonus points for a comment above each
assertQuerysetEqual
that says what we're testing for, and usage ofassertQueries()
as part of that to ensure we're building exactly the queries we want. -
For booleans, we allow
('1', 'true', 'True')
.Not ideal to repeat this everywhere, but let's call that a problem for another day.
- Change Summary:
-
The following changes were made to the
ReviewManager
tests
- UsesassertQueries
instead ofassertNumQueries
- Organized and added comments to the tests - Commits:
-
Summary ID 4fb012c68dda0657cfc1b00636d751405baf6a8a bf75c8ca6cfd33c66bc646a65510865999ff9b7e
Checks run (2 succeeded)
-
-
How should we check that the diffset related to the review is accessible by the user? I see the logic for it in
ReviewRequest._is_diffset_accessible_by(user, diffset)
but how can we efficiently do this in the Managers through a query? -
Should I consolidate these common queries into a variable or something? Then use the variable instead of repeating the full queries
-
This is a lot of work! Thanks for doing this :)
Some small nits and one thing that may be lacking a test.
-
-
-
-
-
For multi-line lists, we generally do:
[ review1, review2, ..., review5, ..., ]
Or, when appropriate, one item per row (generally more appropriate when we may be operating more on the list over time, to minimize changes to the code).
-
-
Another example of where we want periods at the end of the sentence.
Really applies to all such comments.
- Change Summary:
-
- Applied feedback from last review.
- Tuned up the
CommentManager
tests.
- Description:
-
Recently, it was discovered that when querying for reviews there may be
duplicate Q(public=True)
's that appear in the queries to fetch the reviews.The cause of this duplication lies in ReviewManager
's_query
method, whichhandles the logic for querying for reviews. If an unauthenticated or anonymous user was querying for the reviews, the public
query field would be set twice,once to the public
value that was passed and once toTrue
, with the latterensuring that only public reviews get returned to unauthenticated/anonymous users. This however makes the query less efficient and can lead to the wrong results being included in the returned queryset if a False
value forpublic
was passed in. To fix this, we ensure that the
public
query field is only set once, for anytype of user, in ReviewManager._query
. This fix was applied toCommentManager._query
as well.This change also adds a
public
parameter toReviewManager.accessible
toensure that the public
parameter inReviewManager._query
handles all ofthe logic surrounding public and unpublished reviews. Previously, you would have to set the public
query field using theextra_query
parameter inReviewManager.accessible
, which could also lead to duplicate orconflicting Q(public=
queries.+ + Finally, this change also improves the tests for the
ReviewManager
and+ CommentManager
. The tests are more clear, organized and well documented,+ they use assertQueries
, and have more comprehensive test coverage surrounding+ access control. - Commits:
-
Summary ID bf75c8ca6cfd33c66bc646a65510865999ff9b7e edfbeab3d8c44767d124900a020a30af93b5cbf1
- Change Summary:
-
- Removed trailing whitespace
- Added test for
CommentManager
filtering public reviews
- Commits:
-
Summary ID edfbeab3d8c44767d124900a020a30af93b5cbf1 f2598f4dbf985ed743e96b2cd0631e3f8d4ee8b8
Checks run (2 succeeded)
-
Awesome, home stretch. One tiny code styling nit (in a handful of places), but everything else looks ready to go.
-
Missing a trailing comma (we always want to terminate multi-column list items with a trailing comma to minimize line changes if we add to the list).
This applies to other queries as well.