Add support for GitLab API v4
Review Request #9383 — Created Dec. 13, 2017 and submitted
This patch adds support for GitLab API v4 while maintaining backwards
compatability with v3. When we don't know the API version of the host,
we will determine it by optimistically assuming v4 and then falling back
to v3.Additionally, we no longer use the user's password to retrieve an API
token from GitLab. Newer versions (9+) remove the /session API and
require users to generate a token per application in their profile
settings. We now accept this API token directly with instructions how to
create it on new and old versions of GitLab.Some other minor changes were made:
- We now log full stack traces from unexpected Exceptions in the
Repository Form - We now have commit summaries for post commit review for older versions
of GitLab that don't include the entire commit message - Support for nested groups in newer versions of GitLab
- Linked a gitlab.com nested group repository (e.g. group foo/bar, repo
baz) and was able to do post-commit review. - Linked a GitLab 7 group and user repository and was able to do
post-commit review for both. - Ran unit tests.
Description | From | Last Updated |
---|---|---|
Typo in description: GitHub -> GitLab You also have what looks like was supposed to be bullet points but isn't, … |
david | |
Typo in description: "Newer version" -> "Newer versions" |
david | |
GitLab API tests should continue to test API v3, with additional tests for API v4, rather than just moving them … |
chipx86 | |
Make sure you build the docs and check for new errors. There are lots of syntax and reference errors throughout … |
chipx86 | |
Can we keep the v3 API unit tests, and just add new v4 tests, instead of replacing them? |
chipx86 | |
j before l |
david | |
Missing a format specifier in here for message? |
david | |
typo: argsuments |
david | |
Should have a trailing comma |
david | |
Mind adding args/raises to this docstring? |
david | |
Mind adding args/returns/raises? |
david | |
Mind adding args/returns/raises? |
david | |
Mind adding args/returns/raises? |
david | |
Mind adding args/returns/raises? |
david | |
gitlab -> GitLab Should we use double backticks to be consistent with other documentation? |
david | |
Mind adding args/returns/raises? |
david | |
Mind adding returns/raises? |
david | |
Typo: Retur Mind adding returns/raises? |
david | |
Docstring? |
david | |
Mind adding args/returns/raises? |
david | |
returns -> return |
david | |
Otherwise what? |
david | |
returns -> return |
david | |
Docstring? |
david | |
Header keys should be bytestrings |
david | |
Docstring? |
david | |
Should we add some extra context to this log message? |
david | |
This should likely be __repr__ |
david | |
Still missing real docstring content here. |
david | |
typo: succesfully -> successfully |
david | |
Too many blank lines. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
HTTPError is a subclass of URLError, and URLError comes from urllib2. |
david | |
typo: __repr_ -> __repr__ |
david | |
Should be two lines. |
david | |
Typo: Return -> Returns |
david | |
Needs Returns |
david | |
This seems like a bad log message. |
david | |
There's an extra space after "Personal Access" that shouldn't be there. |
david | |
These two lines could be one. |
david | |
Gitlab -> GitLab |
david | |
typo: acount |
david | |
How about wrapping this as: return self._api_get(self.account.hosting_url, 'groups/%s' % group_id)[0] |
david | |
Same here re: wrapping. |
david | |
Header keys should be binary. |
david | |
There's an extra comma at the end of this line, and the ) shouldn't be on the next line. |
david | |
Closing ) from function call shouldn't be on its own line, plus there's an extra comma after the last arg. |
david | |
Trailing ) for function calls shouldn't be on its own line. |
david | |
Indentation got messed up here. |
david | |
Closing ) for function call shouldn't be on its own line. There's also an extra , after the last arg. |
david | |
Needs a trailing comma. |
david | |
Needs a trailing comma. |
david | |
Needs a trailing comma. |
david | |
This should probably be a HostingServiceError. |
chipx86 | |
You should be able to just pass self.causes into the %r directly. It'll repr the list, which will then auto-repr … |
chipx86 | |
Help text shouldn't use <br>. One long string is better here. |
chipx86 | |
Should use double-backticks, since this is rst and not markdown. |
david | |
, unused after the type. |
chipx86 | |
Other docs don't do "Raised if..." That's implied and can be left off. For the last one, "occurred" would be … |
chipx86 | |
"SHA1", and of the blob for the file. |
chipx86 | |
"SHA1" |
chipx86 | |
, unused |
chipx86 | |
Same comment about the "Raised if ..." |
chipx86 | |
Same notes as above. |
chipx86 | |
Same notes as above about "Raised if ..." |
chipx86 | |
While you're here, this should be "Return a list of branches." |
chipx86 | |
This isn't correct. |
chipx86 | |
Same notes as above. |
chipx86 | |
This isn't correct. Shoud be reviewboard.scmtools.core.Commit. |
chipx86 | |
Same notes as above with "Raised if ..." |
chipx86 | |
reviewboard.scmtools.core.Commit |
chipx86 | |
Same notes as above with "Raised if ..." |
chipx86 | |
Can you include the comment from above here as well, about why we're checking both message and title? |
chipx86 | |
Same notes about "Raised if ..." |
chipx86 | |
I assume it's checking both for GitLab API version differences? Can you document those here? |
chipx86 | |
While here, can you add a blank line between these? |
chipx86 | |
Same notes about "Raised if ..." |
chipx86 | |
Same here. |
chipx86 | |
Same here. |
chipx86 | |
:samp: |
chipx86 | |
Will the old version work on the new API? |
chipx86 | |
Typo: "reviewboard" |
chipx86 | |
Same note about "Raised if ..." |
chipx86 | |
Here too. |
chipx86 | |
Same notes about "Raised if ..." |
chipx86 | |
This is a pretty weird and hard-to-read statement, particularly with all the layers of nots and the ^. Given that … |
chipx86 | |
Same comments about "Raised if ..." |
chipx86 | |
"Return the version ..." |
chipx86 | |
Syntax error. Missing the backtick. |
chipx86 | |
I don't see how we'd ever be in a case where the or would occur. If we hit an exception, … |
chipx86 | |
Typo: "arguments" |
chipx86 | |
Same notes about the "Raised if ..." |
chipx86 | |
This should never happen. If it does, it's an internal failure. We should assert. |
chipx86 | |
You can use basestring. |
chipx86 | |
You can use basestring. |
chipx86 | |
I might be wrong about some part of this, but according urlopen docs, the first argument to urlopen is url, … |
chipx86 | |
assertIsNone |
chipx86 | |
This should probably he a HostingServiceError. That way it's properly displayed if it bubbles up. |
chipx86 | |
Mind putting those in alphabetical order? |
chipx86 | |
There's two places where we do this, and have to maintain version compatibility. How about defining a _parse_commit() method that … |
chipx86 | |
Private method, so this should go after all public methods. |
chipx86 | |
And here we are. This method delays building the Commit until the very end, after we've dealt with the initial … |
chipx86 | |
Blank line between these. |
chipx86 | |
There's some old code still here. |
chipx86 | |
Alphabetical order. |
chipx86 | |
Do we have pagination for groups in the new API |
chipx86 | |
Same question about repositories. |
chipx86 | |
Should probably be bytes |
chipx86 | |
E225 missing whitespace around operator |
reviewbot |
- Change Summary:
-
Addressed David's issues
- Description:
-
This patch adds support for GitLab API v4 while maintaining backwards
compatability with v3. When we don't know the API version of the host, we will determine it by optimistically assuming v4 and then falling back to v3. Additionally, we no longer use the user's password to retrieve an API
token from GitLab. Newer version (9+) remove the /session API and require users to generate a token per application in their profile settings. We now accept this API token directly with instructions how to ~ create it on new and old versions of GitHub. ~ create it on new and old versions of GitLab. ~ Some other minor changes were made: * We now log full stack traces from
~ unexpected Exceptions in the Repository Form * We now have commit ~ summaries for post commit review for older versions of GitLab that don't ~ include the entire commit message * Support for nested groups in newer ~ versions of GitLab ~ Some other minor changes were made:
~ ~ - We now log full stack traces from unexpected Exceptions in the
Repository Form
~ - We now have commit summaries for post commit review for older versions
of GitLab that don't include the entire commit message
~ - Support for nested groups in newer versions of GitLab
- We now log full stack traces from unexpected Exceptions in the
- Testing Done:
-
~ - Linked a github.com nested group repository (e.g. group foo/bar, repo
baz) and was able to do post-commit review.
~ - Linked a gitlab.com nested group repository (e.g. group foo/bar, repo
baz) and was able to do post-commit review.
- Linked a GitLab 7 group and user repository and was able to do
post-commit review for both.
- Ran unit tests.
- Linked a github.com nested group repository (e.g. group foo/bar, repo
- Commit:
-
4532cb33c98639a9073f88f51e34b4eb46e074731d885f2bd41d27deac46fa38090cb62067073ef2
Checks run (2 succeeded)
- Change Summary:
-
Addressed David's issues
- Description:
-
This patch adds support for GitLab API v4 while maintaining backwards
compatability with v3. When we don't know the API version of the host, we will determine it by optimistically assuming v4 and then falling back to v3. Additionally, we no longer use the user's password to retrieve an API
~ token from GitLab. Newer version (9+) remove the /session API and ~ token from GitLab. Newer versions (9+) remove the /session API and require users to generate a token per application in their profile settings. We now accept this API token directly with instructions how to create it on new and old versions of GitLab. Some other minor changes were made:
- We now log full stack traces from unexpected Exceptions in the
Repository Form
- We now have commit summaries for post commit review for older versions
of GitLab that don't include the entire commit message
- Support for nested groups in newer versions of GitLab
- We now log full stack traces from unexpected Exceptions in the
- Commit:
-
1d885f2bd41d27deac46fa38090cb62067073ef221571476bea5910b2fe0601f2bdbd52414401ae0
Checks run (2 succeeded)
- Change Summary:
-
Addressed David's issues
- Commit:
-
21571476bea5910b2fe0601f2bdbd52414401ae0c48f6e0ad71c1e68ffd2e2da7a62ec74a7871bd4
Checks run (2 succeeded)
-
-
-
-
-
-
How about wrapping this as:
return self._api_get(self.account.hosting_url, 'groups/%s' % group_id)[0]
-
-
-
-
Closing ) from function call shouldn't be on its own line, plus there's an extra comma after the last arg.
-
-
-
Closing ) for function call shouldn't be on its own line. There's also an extra , after the last arg.
-
-
-
- Change Summary:
-
Addressed David's issues.
- Commit:
-
c48f6e0ad71c1e68ffd2e2da7a62ec74a7871bd4eff43f2f0effa96c3e83a87f4bbe5a039724b7fc
Checks run (2 succeeded)
-
-
GitLab API tests should continue to test API v3, with additional tests for API v4, rather than just moving them all to v4. This ensures we don't regress for users. Might be worth making a test class for each.
-
Make sure you build the docs and check for new errors. There are lots of syntax and reference errors throughout that would come up.
-
-
You should be able to just pass
self.causes
into the%r
directly. It'llrepr
the list, which will then auto-repr
each item in the list. -
-
-
Other docs don't do "Raised if..." That's implied and can be left off.
For the last one, "occurred" would be more correct.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Can you include the comment from above here as well, about why we're checking both
message
andtitle
? -
-
-
-
-
-
-
-
-
-
-
-
-
This is a pretty weird and hard-to-read statement, particularly with all the layers of
not
s and the^
. Given that it's an internal function, and we're responsible for the arguments, I'm not sure we need an exception (which is also not helpful to users if we do screw up with the call and it bubbles up). It's also probably not a big deal if all three are passed in, in practice. How about just doing this further down:if url: assert not hosting_url assert not path else: url = ...
-
-
-
-
I don't see how we'd ever be in a case where the
or
would occur. If we hit an exception, we won't get a value here. If we get a result at all, it's going to have a value, due to the implementation in_try_api_versions
. I want to make sure I'm not missing something. -
-
-
-
-
-
I might be wrong about some part of this, but according
urlopen
docs, the first argument tourlopen
isurl
, not a request.Naming of parameters should also be consistent now, due to more strict kgb internals.
Same with all other fake
_urlopen
functions. -
- Commit:
-
eff43f2f0effa96c3e83a87f4bbe5a039724b7fc203ff4ed40d289eedbc1e7f9d4cb597de13fadac
Checks run (2 succeeded)
-
-
-
-
-
There's two places where we do this, and have to maintain version compatibility. How about defining a
_parse_commit()
method that handles this?I know the other location that does this has a different flow, but I have a thought there. See my next comment.
-
And here we are.
This method delays building the
Commit
until the very end, after we've dealt with the initial data and then generated the diff content. In the case where it's cached, we already have aCommit
object that we throw away, though. So my thought is this that we can do this:if commit is None: ... commit_rsp = self._api_get(...) commit = self._parse_commit(commit_rsp) # Diff generation here... commit.diff = diff return commit
Should simplify much of this code.
-
-
-
-
-
- Commit:
-
4fa838eaf11479a686a607b86098dc4d996a8505a44bb26d988ac615abf53ac8819548f16a9a1fd9