Create token generator classes and a registry for them
Review Request #12415 — Created June 24, 2022 and submitted
Currently, Djblets can only generate SHA1 tokens and token generation is
baked intodjblets.webapi.managers.WebAPITokenManager
. As part of our move to
using an improved token format (API Tokens v2), we need a way to generate
different types of tokens.This change introduces token generators, which handle the logic for token
generation and allows for multiple ways of generating tokens. There are three
classes implemented:
1. A base class.
2. A class for generating our legacy SHA1 tokens.
3. A class for generating our new token format that has a vendor prefix and
checksum.This also adds a registry for the token generators.
- Created unit tests for the token generators.
- Ran all unit tests in
djblets.secrets.tests
.
Summary | ID |
---|---|
069535f539beaea702acdbf986f55404f71da379 |
Description | From | Last Updated |
---|---|---|
line too long (80 > 79 characters) Column: 80 Error code: E501 |
reviewbot | |
'django.contrib.auth.models.User' imported but unused Column: 1 Error code: F401 |
reviewbot | |
line too long (80 > 79 characters) Column: 80 Error code: E501 |
reviewbot | |
'django.db.models' imported but unused Column: 1 Error code: F401 |
reviewbot | |
'django.db.models' imported but unused Column: 1 Error code: F401 |
reviewbot | |
Docstrings in this file should be "Testing ..." |
david | |
Docstrings in this file should be "Testing ..." You can also put the """ on the same line for this … |
david | |
I also mentioned it in the review for /r/12419, but we should instantiate the registry as an attribute of this … |
david | |
Typo: NotImeplementedError -> NotImplementedError. |
david | |
There's an extra blank line here. |
david | |
Typo: NotImeplementedError -> NotImplementedError. |
david | |
There's not actually a **kwargs in the method signature. |
david | |
line too long (81 > 79 characters) Column: 80 Error code: E501 |
reviewbot | |
line too long (80 > 79 characters) Column: 80 Error code: E501 |
reviewbot | |
line too long (89 > 79 characters) Column: 80 Error code: E501 |
reviewbot | |
For these tests, you can probably just put the data within the call. Helps keep the construction and parameters together. … |
chipx86 | |
This can be consolidated: token = ( '_%s' % self.token_generator.create_token(...)[1:] ) |
chipx86 | |
Typo: "charcter" -> "character" |
chipx86 | |
This can be consolidated: token = ( '!%s' % self.token_generator.create_token(...)[1:] ) |
chipx86 | |
This should have a doc comment. |
chipx86 | |
Missing , optional |
chipx86 | |
This method is missing a Returns. |
chipx86 | |
This should make it clear what the return type should be if a determination could not be made. |
chipx86 | |
Missing , optional |
chipx86 | |
Let's say "legacy SHA1 tokens". |
chipx86 | |
Let's say "legacy SHA1 tokens". |
chipx86 | |
These should be in alphabetical order. |
chipx86 | |
This method doesn't seem to ever return None. However, it does raise an exception, which should be documented. |
chipx86 | |
Blank line between these. |
chipx86 | |
Localized text can't be formatted within the _(...). The % must be after. |
chipx86 | |
No need for the outer parens. Strings generally concatenate faster if joining via a format string, rather than +. So: … |
chipx86 | |
The in check requires a loop through charset for each character in token. Two things can be done to speed … |
chipx86 | |
For Registry, get() will raise an exception if the entry is not found. |
chipx86 | |
We usually use snake_case for identifiers. |
chipx86 | |
Doc comments are in the form of: #: Summary #: #: ... Same conventions as docstrings. These will get processed … |
chipx86 | |
token_info probably shouldn't be optional here, should it? If so, then the keys within must also be optional, and the … |
chipx86 | |
For Raises, the description must be indented. |
chipx86 | |
Let's avoid the repeated self.CHARSET attribute lookup by pulling that out first. |
chipx86 | |
Rather than re-assigning, you can do: checksum = ( self._base62_encode(...) .zfill(...) ) |
chipx86 | |
This can be one statement. |
chipx86 | |
Same note about token_info. This will apply to all other classes too. |
chipx86 | |
Same as with the SHA1 generator, we should avoid a search of [0-9a-f_] for every character in the hash. We … |
chipx86 | |
We access self.CHARSET many times, each involving an attribute lookup, which isn't free. Let's pull that out into a local … |
chipx86 | |
Slightly faster as: return ''.join(reversed(result)) This loops through result once. If using .reverse() instead, it loops twice, once to reverse … |
chipx86 | |
Blank line between these. |
chipx86 | |
The formatting has to happen after _(...). |
chipx86 | |
For these tests, let's feed in a hard-coded token. This gives us clear test data to work with, and avoids … |
chipx86 | |
Should be fully lowercase (snake_case is typically lowercase). |
chipx86 | |
As this is a caller-time problem, we probably don't need to log anything. The exception will be fine. Same with … |
chipx86 | |
Small nit. We usually keep the % on the line with the argument. |
chipx86 | |
If this is missing content, we probably just want to say "Nope, not valid for this generator" and return False. |
chipx86 |
- Change Summary:
-
- Adds a registry for token generators
- Add tests for KeyErrors
- Summary:
-
[WIP] Create Token Generator Classes[WIP] Create token generator classes and a registry for them
- Description:
-
~ Currently, Djblets can only generate SHA1 tokens. As part of our move to using
~ an improved token format, we need a way to generate different types of tokens. ~ Currently, Djblets can only generate SHA1 tokens and token generation is
~ baked into djblets.webapi.managers.WebAPITokenManager
. As part of our move to+ using an improved token format (API Tokens v2), we need a way to generate + different types of tokens. ~ This change introduces Token Generator classes, with three classes implemented:
~ 1. A Base class ~ 2. A class for our legacy SHA1 tokens ~ 3. A class for our new token format that has a vendor prefix and checksum. ~ This change introduces token generators, which handle the logic for token
~ generation and allows for multiple ways of generating tokens. There are three ~ classes implemented: ~ 1. A base class. + 2. A class for generating our legacy SHA1 tokens. + 3. A class for generating our new token format that has a vendor prefix and + checksum. + + This also adds a registry for the token generators.
- Commits:
-
Summary ID 548aa507d38225ad8858092b4997b1cbe494729e 3766327e9807f31917daddee7eb196ce721b77a7 - Depends On:
-
- Diff:
Revision 2 (+978)
- Change Summary:
-
removed unused imports
- Testing Done:
-
+ - Created unit tests for the token generators.
+ - Ran all unit tests with success.
+ - Still need to add unit tests for token generators'
validate_token
method.
- Commits:
-
Summary ID 3766327e9807f31917daddee7eb196ce721b77a7 b67fa67e06c43ca27c710ada222a14bf32d092e1 - Diff:
-
Revision 3 (+972)
Checks run (2 succeeded)
- Change Summary:
-
- Implemented
validate_token
for token generators - Finished the unit tests
- Implemented
- Commits:
-
Summary ID b67fa67e06c43ca27c710ada222a14bf32d092e1 a0e83e27e32db959fdeab953c795d2a54f468082 - Diff:
-
Revision 4 (+1098)
Checks run (2 succeeded)
- Change Summary:
-
- Removed a
valid_prefixes
attribute for theVendorChecksumTokenGenerator
so that I don't have to subclass this class in Reviewboard. - Added back the
token_info
parameter to thevalidate_token
functions, so that we can pass in which prefixes are valid.
- Removed a
- Commits:
-
Summary ID a0e83e27e32db959fdeab953c795d2a54f468082 658228f96385f1af8073c6edb06fa3fa93da394f - Diff:
-
Revision 5 (+1146)
Checks run (2 succeeded)
- Change Summary:
-
- Fixed test docstrings.
- Made the token generator registry an attribute of the token generators module.
- Testing Done:
-
- Created unit tests for the token generators.
~ - Ran all unit tests with success.
~ - Ran all unit tests in
djblets.secrets.tests
.
- - Still need to add unit tests for token generators'
validate_token
method.
- Commits:
-
Summary ID 658228f96385f1af8073c6edb06fa3fa93da394f c407408eb185f52c364fba90678a6f58f8b33016 - Diff:
-
Revision 6 (+1144)
- Change Summary:
-
Fixed some imports and docstrings being too long
- Commits:
-
Summary ID c407408eb185f52c364fba90678a6f58f8b33016 caa42c5ea2fc4c2b5e0710d4937ef97561082609 - Diff:
-
Revision 7 (+1152)
Checks run (2 succeeded)
-
-
For these tests, you can probably just put the data within the call. Helps keep the construction and parameters together.
Same below.
-
-
-
-
-
-
-
-
-
-
-
-
This method doesn't seem to ever return
None
.However, it does raise an exception, which should be documented.
-
-
-
No need for the outer parens.
Strings generally concatenate faster if joining via a format string, rather than
+
. So:prefix = '%s%s%s' % (prefix, str(attempt), timezone.now().isoformat())
-
The
in
check requires a loop throughcharset
for each character intoken
. Two things can be done to speed that up:charset = set('0123456789abcdef')
Or, probably better, define a regex for what a SHA1 token can be:
[0-9A-Fa-f]{40}
This would allow for a single optimized check.
-
-
-
Doc comments are in the form of:
#: Summary #: #: ...
Same conventions as docstrings. These will get processed by Sphinx.
-
token_info
probably shouldn't be optional here, should it?If so, then the keys within must also be optional, and the function must check for them.
-
-
-
-
-
-
Same as with the SHA1 generator, we should avoid a search of
[0-9a-f_]
for every character in the hash.We also want the cheapest check first, which would be the length check, since that's stored along with a string.
So I'd suggest a regex just for the hash part on the class-level, as simply
^_[0-9a-f]+$
, and a check like:return (len(token) == 255 and HASH_RE.match(token[len(token_type):]))
-
We access
self.CHARSET
many times, each involving an attribute lookup, which isn't free. Let's pull that out into a local variable. -
Slightly faster as:
return ''.join(reversed(result))
This loops through
result
once. If using.reverse()
instead, it loops twice, once to reverse the list, the second time to join items. -
-
- Commits:
-
Summary ID caa42c5ea2fc4c2b5e0710d4937ef97561082609 a02662bff38fabcdc8c69fc3ca6cae745af32516 - Diff:
-
Revision 8 (+1198)
Checks run (2 succeeded)
- Change Summary:
-
Adds a central configuration for the default token generator and a method for getting it from the registry
- Commits:
-
Summary ID a02662bff38fabcdc8c69fc3ca6cae745af32516 68fdeea6caf901651cfba389d452324b06a3cfcf - Diff:
-
Revision 9 (+1236)
Checks run (2 succeeded)
-
Just a few things left that I noticed this round. Almost there!
-
For these tests, let's feed in a hard-coded token. This gives us clear test data to work with, and avoids side effects from
create_token
changes or regressions.This applies to the other generator tests as well.
-
-
As this is a caller-time problem, we probably don't need to log anything. The exception will be fine.
Same with the other generator.
-
-
If this is missing content, we probably just want to say "Nope, not valid for this generator" and return
False
.
- Commits:
-
Summary ID 68fdeea6caf901651cfba389d452324b06a3cfcf 069535f539beaea702acdbf986f55404f71da379 - Diff:
-
Revision 10 (+1198)