Add an integration for Travis CI.

Review Request #8921 — Created May 5, 2017 and submitted

Information

rbintegrations
master
c032765...

Reviewers

This change adds a new integration which allows people to build prospective
changes via Travis CI.

The Travis CI API allows triggering a build in a way that overrides the config
from the repository's .travis.yml file. We use this to send a new
configuration which knows how to apply the patch and report the results back to
Review Board.

The configuration for the integration allows users to set the conditions
normally, with the exception that the choices for the "Repository" condition
are limited to only show GitHub repositories (since Travis CI doesn't work with
anything else). It also requires selecting which Travis endpoint to use (either
travis-ci.org for open-source projects, travis-ci.com for private projects, or
an enterprise server with a custom domain), along with providing an API token.

Currently, builds show up under the main "Branches" section of the repository
in the Travis CI user interface. Because this can pollute things (for example,
showing that the master branch is broken because a build of a review request
failed), for now I'm recommending that people create a fake review-requests
branch on their GitHub remote and then set that branch name in the integration
config. This isn't perfect, but it at least keeps review requests builds
separate. I'm starting to talk to the Travis CI folks to see if we can get some
more control through the API, perhaps even letting us list these builds in the
"Pull Requests" section instead of the "Branches" section.

The modifications made to the user-entered configuration involve changing the
script section to insert a few commands to check out the correct parent
revision and apply the patch, and modifications to the notifications section
to disable e-mail notification and set up a webhook to receive results. For
example, if the build config was set to the following:

language: python
python:
    - 2.7
env:
    - DJANGO_VERSION=1.6.11
    - DJANGO_VERSION=1.8.2
install:
    - python setup.py develop
    - pip install -r dev-requirements.txt
script:
    - python ./tests/runtests.py

Then the resulting configuration that would be sent to Travis would look
something like this:

language: python
python:
    - 2.7
env:
    global:
        - REVIEWBOARD_STATUS_UPDATE_ID=71
        - REVIEWBOARD_TRAVIS_INTEGRATION_CONFIG_ID=4
    matrix:
        - DJANGO_VERSION=1.6.11
        - DJANGO_VERSION=1.8.2
install:
    - python setup.py develop
    - pip install -r dev-requirements.txt
merge_mode: replace
notifications:
    email: false
    webhooks:
        on_start: always
        urls:
            - http://reviewboard.example.com/rbintegrations/travis-ci/webhook/
script:
    - git fetch --unshallow origin
    - git checkout 5ec2912cd1cad96a3a20ea5857e9ed2d1de0d465
    - echo <big base64 blob> | base64 --decode | patch -p1
    - python ./tests/runtests.py

Integration configuration:

  • Tested that only GitHub repositories showed up under the "Repository"
    condition.
  • Tested interaction where the server name field was hidden by default and only
    shown when the "Travis CI" field was set to "Enterprise (custom domain)".
  • Verified that a correct error message was shown if the API token was
    incorrect.
  • Verified that an incorrect configuration would show the proper errors as
    reported by the Travis CI lint API.

Usage:

  • Posted a variety of changes which should either fail or pass the build, and
    saw the results reported correctly.
  • Verified that webhook signature validation was correct, passing in regular
    use and failing if I made any changes to the payload contents.

Description From Last Updated

Small typo in the example generated configuration. The comma shouldn't be there after the DJANGO_VERSION=1.6.11.

chipx86chipx86

This help text should probs be aligned with the description above.

brenniebrennie

Alignment here is off.

brenniebrennie

We definitely should have a warning that they should omit secret environment variables, because people will be able to print …

brenniebrennie

Col: 15 'selectedEndpoint' is defined but never used.

reviewbotreviewbot

'django.utils.six.moves.urllib.error.URLError' imported but unused

reviewbotreviewbot

local variable 'url' is assigned to but never used

reviewbotreviewbot

'django.core.exceptions.ValidationError' imported but unused

reviewbotreviewbot

local variable 'section_name' is assigned to but never used

reviewbotreviewbot

'json' imported but unused

reviewbotreviewbot

local variable 'review_request_id' is assigned to but never used

reviewbotreviewbot

local variable 'data' is assigned to but never used

reviewbotreviewbot

Any reason you don't use an arrow func here?

brenniebrennie

Module docstring

brenniebrennie

Can we not just use the one from hostingsvcs to avoid duplicate code?

brenniebrennie

"HTTP GET"

brenniebrennie

Should be indented.

brenniebrennie

Needs to be qualified.

brenniebrennie

Should be indented.

brenniebrennie

Needs to be qualified.

brenniebrennie

Needs to be qualified.

brenniebrennie

Needs to be qualified.

brenniebrennie

Should be indented.

brenniebrennie

Needs to be qualified.

brenniebrennie

Should be indented.

brenniebrennie

Module level docstring.

brenniebrennie

Don't we usually have these at top of funcs?

brenniebrennie

Django behaviour is to remove error'ed fields from cleaned_data. Shouldn't this be a ValidationError(_(...)) since self.error_class is ErrorList ? Same …

brenniebrennie

Module level docstring.

brenniebrennie

Can we distribute these?

brenniebrennie

Rather, review requests that are attachment-only.

brenniebrennie

This isn't Review Bot.

brenniebrennie

Module level docstring.

brenniebrennie

No blank line here.

brenniebrennie

Blank line between these.

brenniebrennie

W503 line break before binary operator

reviewbotreviewbot

You still need to pop travis_custom_endpoint out of self._errors. Same below.

brenniebrennie

E127 continuation line over-indented for visual indent

reviewbotreviewbot

E501 line too long (83 > 79 characters)

reviewbotreviewbot

W503 line break before binary operator

reviewbotreviewbot

E127 continuation line over-indented for visual indent

reviewbotreviewbot

E501 line too long (83 > 79 characters)

reviewbotreviewbot

Can we use travis-ci instead of travisci?

chipx86chipx86

You can do: $server.setVisible($endpoint.val() === 'E');

chipx86chipx86

Maybe condense these to be two lines? u = urlopen(URLRequest(...)) return json.loads(u.read())

chipx86chipx86

dict?

chipx86chipx86

reviewboard.reviews.models.review_request.ReviewRequest

chipx86chipx86

Maybe we can use the existing list and perform a - [ReviewRequestRepositoryTypeChoice] (or whatever)?

chipx86chipx86

and on the previous line.

chipx86chipx86

W503 line break before binary operator

reviewbotreviewbot

We should probably use six.text_type(e) instead of e directly. Same below.

chipx86chipx86

Can simplify this a bit: if warning['key']: ... message = ... else: message = warning['message'] self._errors['travis_yml'] = self.error_class([message])

chipx86chipx86

Can we log with request=self.request on all these logging statements, so we'll have user/Local Site context?

chipx86chipx86

repository's Let's also just use double quotes for this string instead of escaping the apostrophes.

chipx86chipx86

Let's pull out the repository into a variable. It'll prevent the Review Bot complaint, since it should be able to …

chipx86chipx86

E127 continuation line over-indented for visual indent

reviewbotreviewbot

We can include this in the function.

chipx86chipx86

Can these be: notifications = travis_config.get('notifications') or {} ?

chipx86chipx86

Can use += [ ... ]

chipx86chipx86

We're going to need to check for IntegrityErrors, in case another process beat this one to the punch on creating …

chipx86chipx86

*args

chipx86chipx86

**kwargs

chipx86chipx86

E501 line too long (83 > 79 characters)

reviewbotreviewbot

You can wrap after the . anywhere now.

chipx86chipx86

This will crash if HTTP_SIGNATURE isn't present.

chipx86chipx86

Should be bullet-proof this? Log an error if we get a KeyError?

chipx86chipx86

Does this raise an exception if any part of this fails?

chipx86chipx86

We should bullet-proof this as well.

chipx86chipx86

W503 line break before binary operator

reviewbotreviewbot

E501 line too long (81 > 79 characters)

reviewbotreviewbot

E501 line too long (83 > 79 characters)

reviewbotreviewbot

E122 continuation line missing indentation or outdented

reviewbotreviewbot

You can pass the method in directory. There's no this being used.

chipx86chipx86

We should guard against the request failing or the data not being parsable. (Proxy servers may get in the way …

chipx86chipx86

"GitHub"

chipx86chipx86

GitHub

brenniebrennie

Can you wrap these in parens so it's a little more visually clear what's happening? Since this is Python 2.7, …

chipx86chipx86

W503 line break before binary operator

reviewbotreviewbot

Should this be marked for translation?

brenniebrennie

Do we want to limit the scope of this token?

brenniebrennie

E501 line too long (83 > 79 characters)

reviewbotreviewbot

Sine this line's too long, and we dig into cryptography.hazmat.primitives a bunch, let's just pull primitives out.

chipx86chipx86

W503 line break before binary operator

reviewbotreviewbot
Checks run (2 failed, 1 failed with error)
JSHint failed.
PEP8 Style Checker internal error.
Pyflakes failed.

JSHint

Pyflakes

david
brennie
  1. 
      
  2. Show all issues

    Any reason you don't use an arrow func here?

    1. There's no real reason to use it (we don't care about lexical this and it doesn't need to be that compact), so we might as well avoid the extra overhead added by the transpilation.

  3. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Module docstring

  4. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Can we not just use the one from hostingsvcs to avoid duplicate code?

    1. It seemed super weird to be to be using something which is specific to hosting services inside this. If you moved it somewhere more common (djblets?) I'd be happy to use it.

    2. Yeah, at this point it should probably live there (and mayb be renamed to just Request instead of URLRequest)

  5. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    "HTTP GET"

  6. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Should be indented.

  7. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Needs to be qualified.

  8. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Should be indented.

  9. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Needs to be qualified.

  10. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Needs to be qualified.

  11. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Needs to be qualified.

  12. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Should be indented.

  13. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Needs to be qualified.

  14. rbintegrations/travisci/api.py (Diff revision 2)
     
     
    Show all issues

    Should be indented.

  15. rbintegrations/travisci/forms.py (Diff revision 2)
     
     
    Show all issues

    Module level docstring.

  16. rbintegrations/travisci/forms.py (Diff revision 2)
     
     
    Show all issues

    Don't we usually have these at top of funcs?

    1. No? I prefer to import things adjacent to where they're used.

  17. rbintegrations/travisci/forms.py (Diff revision 2)
     
     
    Show all issues

    Django behaviour is to remove error'ed fields from cleaned_data.

    Shouldn't this be a ValidationError(_(...)) since self.error_class is ErrorList ?

    Same below.

    1. Based on the docs and functionality, this is correct. The normal form class catches ValidationError and extracts the message to put it into the ErrorList.

  18. rbintegrations/travisci/integration.py (Diff revision 2)
     
     
    Show all issues

    Module level docstring.

  19. rbintegrations/travisci/integration.py (Diff revision 2)
     
     
     
    Show all issues

    Can we distribute these?

    1. Yep, so long as we follow their normal branding guidelines.

  20. rbintegrations/travisci/integration.py (Diff revision 2)
     
     
    Show all issues

    Rather, review requests that are attachment-only.

    1. That's not the case. We already checked that the review request has a repository. In this case we're checking that if it's an update, that the update added a new diff.

    2. Yes but wouldn't get_latest_diffset() return the latest diffset, not the diffset from the latest change? ie, get_latest_diffset() is None should be mutually exclusive with review_requet.repository being not None.

      The change for the latest change having a diff comes after when you test for 'diff' in fields_changed.

    3. I made this comment better in the latest diff.

  21. rbintegrations/travisci/integration.py (Diff revision 2)
     
     
     
     
    Show all issues

    This isn't Review Bot.

  22. rbintegrations/travisci/views.py (Diff revision 2)
     
     
    Show all issues

    Module level docstring.

  23. rbintegrations/travisci/views.py (Diff revision 2)
     
     
    Show all issues

    No blank line here.

  24. rbintegrations/travisci/views.py (Diff revision 2)
     
     
     
    Show all issues

    Blank line between these.

  25. 
      
david
Review request changed
Commit:
c0112d02f0b7c92c854ffc9a1076ba2d0b916bc5
8fd69d70f07b57c21ad8733c1c04ae604d21493f

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

brennie
  1. 
      
  2. rbintegrations/travisci/forms.py (Diff revision 3)
     
     
    Show all issues

    You still need to pop travis_custom_endpoint out of self._errors. Same below.

    1. Rather, out of self.cleaned_data. Haven't had my coffee yet.

    2. Why is that? This doesn't do any additional processing on cleaned_data once validation fails.

  3. 
      
david
Review request changed
Change Summary:

Some random fixes and improvements that came up as I was writing tests.

Commit:
8fd69d70f07b57c21ad8733c1c04ae604d21493f
07a5b58bfa7efe5c5880dec5995ac12f557383bc

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

david
Review request changed
Commit:
07a5b58bfa7efe5c5880dec5995ac12f557383bc
0f1422b22a439a8e6a89032ab25d4e78500c09bf

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

chipx86
  1. 
      
  2. Show all issues

    Small typo in the example generated configuration. The comma shouldn't be there after the DJANGO_VERSION=1.6.11.

  3. rbintegrations/extension.py (Diff revision 5)
     
     
    Show all issues

    Can we use travis-ci instead of travisci?

    1. Given that a lot of the other filenames need to be "travisci" (python modules, etc), I'd rather keep them all consistent.

  4. Show all issues

    You can do:

    $server.setVisible($endpoint.val() === 'E');
    
  5. rbintegrations/travisci/api.py (Diff revision 5)
     
     
     
     
     
    Show all issues

    Maybe condense these to be two lines?

    u = urlopen(URLRequest(...))
    return json.loads(u.read())
    
  6. rbintegrations/travisci/api.py (Diff revision 5)
     
     
     
    Show all issues

    dict?

  7. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
    Show all issues

    reviewboard.reviews.models.review_request.ReviewRequest

  8. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
     
     
     
     
     
     
     
     
    Show all issues

    Maybe we can use the existing list and perform a - [ReviewRequestRepositoryTypeChoice] (or whatever)?

  9. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
     
    Show all issues

    and on the previous line.

    1. Normally yes but that would mean wrapping things in a much more ugly way. This is a lot more readable.

    2. We're going to get the error every single time this file goes through review, which is annoying. Let's just pull out stuff as variables and shrink the conditional.

  10. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
    Show all issues

    We should probably use six.text_type(e) instead of e directly.

    Same below.

  11. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
     
     
     
    Show all issues

    Can simplify this a bit:

    if warning['key']:
        ...
    
        message = ...
    else:
        message = warning['message']
    
    self._errors['travis_yml'] = self.error_class([message])
    
  12. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
     
     
    Show all issues

    Can we log with request=self.request on all these logging statements, so we'll have user/Local Site context?

  13. rbintegrations/travisci/forms.py (Diff revision 5)
     
     
    Show all issues

    repository's

    Let's also just use double quotes for this string instead of escaping the apostrophes.

  14. rbintegrations/travisci/integration.py (Diff revision 5)
     
     
     
     
     
    Show all issues

    Let's pull out the repository into a variable. It'll prevent the Review Bot complaint, since it should be able to wrap, and we access the repository further below anyway.

  15. rbintegrations/travisci/integration.py (Diff revision 5)
     
     
    Show all issues

    We can include this in the function.

  16. rbintegrations/travisci/integration.py (Diff revision 5)
     
     
     
     
     
     
     
     
     
     
    Show all issues

    Can these be:

    notifications = travis_config.get('notifications') or {}
    

    ?

  17. rbintegrations/travisci/integration.py (Diff revision 5)
     
     
     
     
     
    Show all issues

    Can use += [ ... ]

  18. rbintegrations/travisci/integration.py (Diff revision 5)
     
     
     
     
     
    Show all issues

    We're going to need to check for IntegrityErrors, in case another process beat this one to the punch on creating the user.

  19. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    *args

  20. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    **kwargs

  21. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    You can wrap after the . anywhere now.

  22. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    This will crash if HTTP_SIGNATURE isn't present.

  23. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    Should be bullet-proof this? Log an error if we get a KeyError?

  24. rbintegrations/travisci/views.py (Diff revision 5)
     
     
     
    Show all issues

    Does this raise an exception if any part of this fails?

  25. rbintegrations/travisci/views.py (Diff revision 5)
     
     
    Show all issues

    We should bullet-proof this as well.

  26. 
      
david
Review request changed
Description:
   

This change adds a new integration which allows people to build prospective

    changes via Travis CI.

   
   

The Travis CI API allows triggering a build in a way that overrides the config

    from the repository's .travis.yml file. We use this to send a new
    configuration which knows how to apply the patch and report the results back to
    Review Board.

   
   

The configuration for the integration allows users to set the conditions

    normally, with the exception that the choices for the "Repository" condition
    are limited to only show GitHub repositories (since Travis CI doesn't work with
    anything else). It also requires selecting which Travis endpoint to use (either
    travis-ci.org for open-source projects, travis-ci.com for private projects, or
    an enterprise server with a custom domain), along with providing an API token.

   
   

Currently, builds show up under the main "Branches" section of the repository

    in the Travis CI user interface. Because this can pollute things (for example,
    showing that the master branch is broken because a build of a review request
    failed), for now I'm recommending that people create a fake review-requests
    branch on their GitHub remote and then set that branch name in the integration
    config. This isn't perfect, but it at least keeps review requests builds
    separate. I'm starting to talk to the Travis CI folks to see if we can get some
    more control through the API, perhaps even letting us list these builds in the
    "Pull Requests" section instead of the "Branches" section.

   
   

The modifications made to the user-entered configuration involve changing the

    script section to insert a few commands to check out the correct parent
    revision and apply the patch, and modifications to the notifications section
    to disable e-mail notification and set up a webhook to receive results. For
    example, if the build config was set to the following:

   
   

   
   
language: python
   
python:
   
    - 2.7
   
env:
   
    - DJANGO_VERSION=1.6.11
   
    - DJANGO_VERSION=1.8.2
   
install:
   
    - python setup.py develop
   
    - pip install -r dev-requirements.txt
   
script:
   
    - python ./tests/runtests.py
   
   
   

   
   

Then the resulting configuration that would be sent to Travis would look

    something like this:

   
   

   
   
language: python
   
python:
   
    - 2.7
   
env:
   
    global:
   
        - REVIEWBOARD_STATUS_UPDATE_ID=71
   
        - REVIEWBOARD_TRAVIS_INTEGRATION_CONFIG_ID=4
   
    matrix:
~  
        - DJANGO_VERSION=1.6.11,
  ~
        - DJANGO_VERSION=1.6.11
   
        - DJANGO_VERSION=1.8.2
   
install:
   
    - python setup.py develop
   
    - pip install -r dev-requirements.txt
   
merge_mode: replace
   
notifications:
   
    email: false
   
    webhooks:
   
        on_start: always
   
        urls:
   
            - http://reviewboard.example.com/rbintegrations/travis-ci/webhook/
   
script:
   
    - git fetch --unshallow origin
   
    - git checkout 5ec2912cd1cad96a3a20ea5857e9ed2d1de0d465
   
    - echo <big base64 blob> | base64 --decode | patch -p1
   
    - python ./tests/runtests.py
   
-  
-  

-  
-  

Documentation and unit tests will be forthcoming in future changes.

Commit:
0f1422b22a439a8e6a89032ab25d4e78500c09bf
d2840643f6e929a6da5872400244fe9ebeb3e5e7

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

david
Review request changed
Commit:
d2840643f6e929a6da5872400244fe9ebeb3e5e7
31e510562cf021a7d67cb92dadfc23adad7ccde2

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

chipx86
  1. 
      
  2. Show all issues

    You can pass the method in directory. There's no this being used.

  3. rbintegrations/travisci/api.py (Diff revision 7)
     
     
    Show all issues

    We should guard against the request failing or the data not being parsable. (Proxy servers may get in the way and spit out garbage, for instance.)

    What we do in GitHub and other places is we have a function that does the request and returns JSON data, and it checks everything. We should maybe have _make_request do that (optionally, if it's being used for non-JSON requests as well).

    1. The caller has error handling for all of this. I'll clarify that this can raise other errors.

  4. rbintegrations/travisci/forms.py (Diff revision 7)
     
     
    Show all issues

    "GitHub"

  5. rbintegrations/travisci/forms.py (Diff revision 7)
     
     
     
     
     
     
    Show all issues

    Can you wrap these in parens so it's a little more visually clear what's happening?

    Since this is Python 2.7, we can also use {...} instead of set([...]}.

  6. rbintegrations/travisci/views.py (Diff revision 7)
     
     
    Show all issues

    Sine this line's too long, and we dig into cryptography.hazmat.primitives a bunch, let's just pull primitives out.

  7. 
      
brennie
  1. 
      
  2. rbintegrations/travisci/forms.py (Diff revision 7)
     
     
    Show all issues

    GitHub

  3. rbintegrations/travisci/integration.py (Diff revision 7)
     
     
    Show all issues

    Should this be marked for translation?

    1. We don't currently translate the descriptions for the other integrations. Maybe we should, but if so we'll probably want to audit the whole codebase for i18n-related things.

  4. rbintegrations/travisci/integration.py (Diff revision 7)
     
     
    Show all issues

    Do we want to limit the scope of this token?

    1. We don't actually need the API token for travis. I'll get rid of this.

  5. 
      
david
Review request changed
Commit:
31e510562cf021a7d67cb92dadfc23adad7ccde2
958654befc4863eb1613d1959f8120f08f3ea036

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

david
brennie
  1. Barring the UI issues below, this all looks good to me.

  2. Show all issues

    This help text should probs be aligned with the description above.

    1. This is style stuff in RB itself. Shouldn't be fixed as part of this change.

  3. Show all issues

    Alignment here is off.

  4. 
      
brennie
  1. 
      
  2. Show all issues

    We definitely should have a warning that they should omit secret environment variables, because people will be able to print out the environment in the log with a malicious patch.

  3. 
      
david
Review request changed
Status:
Completed
Change Summary:
Pushed to master (63b89ca)