Enabling webhooks to close review-requests after commits

Review Request #9585 — Created Feb. 4, 2018 and updated

Information

rb-gateway
master
944a593...

Reviewers

Review request for feature of rb-gateway to support webhooks on git events. The workflow is
1) Once rb-gateway runs, it installs hookscripts in the repos' it manages. It will install the hookscript under: path/to/repo/.git/hooks/<event-type>.d/99-rb-gateway.sh in addition to a <event-type> script under path/to/repo/.git/hooks/ that runs any pre-existing <event-type> scripts in the repo. We only support the post-commit event for now.
2) Make a commit in the repo hosted on rb-gateway. This will run the hookscript installed in Step 1
3) Hookscript will run which will pipe the commit data from Step 2 to rb-gateway and rb-gateway will make an HTTP request to the reviewboard instance (assuming this is properly configured in config.json)
4) Reviewboard will receive this HTTP request, and close the corresponding review

Between the first diff and the latest one, there's been a lot of file movement since I was working off a patch link r/9402 but there was a non-trivial rewrite of rb-gateway in between on master.

Changes include:
1) Updating README on how to run rbgateway
2) Adding API calls for changing the config through http requests rather than having to change the file itself(i.e. DeleteWebhook DisableWebhook RegisterWebhook)
3) Default hookscripts included in rb-gateway, rb-gateway will automatically install them based on ./config/post-commit.sh (or whatever future event-type we will support)
4) Rb-gateway sending an HTTP POST to the webhook urls

Corresponding review for reviewboard side of the code:
https://reviews.reviewboard.org/r/9700/

Ran tests

Manual test:
1) Make review request on locally running reviewboard instance of repo rb-gateway will be managing
2) Run rb-gateway with config pointing to a local repo and a webhook
3) Make commit in local repo

Review request in Step 1 is closed after Step 3 has completed.

Description From Last Updated

Remove this.

brenniebrennie

So the method you're doing here will work, but it does modify existing files (which we probably do not want …

brenniebrennie

Git hooks should be named post-commit, etc. without the .sh.

brenniebrennie

Should the last character (',') be truncated from the error message?

jshephardjshephard

Are their constants/enums for HTTP methods somewhere?

rpolyanorpolyano

Is there a way to not hardcode this here? Perhaps move it somewhere else in (or out of) the code?

rpolyanorpolyano

Can you rewrite the summary in the imperative mood, e.g.: // Register a wehook.

brenniebrennie

This should return 201 Created.

brenniebrennie

Likewise here.

brenniebrennie

Unnecessary, as thats the default behaviour of the HTTP standard.

brenniebrennie

Care to put the body param before the t?

brenniebrennie

} on next line.

brenniebrennie

Is that a single space or is it a tab character? IMO we should use two (my preference) or four …

brenniebrennie

Missing a period.

brenniebrennie

I think we should either manage webhooks exclusively via the API or exclusively via config.json. If the former, we should …

brenniebrennie

We really just need the ID. Everything else we can get if we need it.

brenniebrennie

Comments should be in proper sentences beginning with capitals and ending with periods. Same goes for all comments throughout.

brenniebrennie

Can you write your doc-comments in the imperative mood, e.g. "Return whether or not the webhook has valid fields."

brenniebrennie

remove this line.

brenniebrennie

Can be one statement

brenniebrennie

Ideally we can have this as a template string inside go and then replace (1) the path to rbgateway and …

brenniebrennie

So this isn't strictly correct if you're not pushing to master. You will want to read from standard input the …

brenniebrennie

This should live in the hooks package

brenniebrennie

Can we move the code thats executed when triggeredByHookscript is true into another function? That way we can do: if …

brenniebrennie

Testing code?

brenniebrennie

You should use "${HOOK}" here in case it contains a space.

brenniebrennie

No blank line here.

brenniebrennie

I've never read or worked with Go before, but I think it's pretty cool how readable some of it is. …

JT jtrang

Another question, what does the syntax for the parameter t *testing.T mean?

JT jtrang

I'm not familiar with go lang at all really so I'm just wondering, how come sometimes you use "if err …

JC jcorrive
FL
FL
FL
brennie
  1. 
      
  2. git_repository.go (Diff revision 4)
     
     
    Show all issues

    Remove this.

  3. git_repository.go (Diff revision 4)
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
    Show all issues

    So the method you're doing here will work, but it does modify existing files (which we probably do not want to do).

    Instead how about the following approach:

    1. Make a new directory .git/hooks/{SCRIPT_TYPE}.d, where SCRIPT_TYPE is e.g., post-commit
    2. If .git/hooks/{SCRIPT_TYPE} exists, move it to .git/hooks/{SCRIPT_TYPE}.d/00-original
    3. Install our hook script as .git/hooks/{SCRIPT_TYPE}.d/99-rbgateway:
    4. Create a new .git/hooks/{SCRIPT_TYPE} with the following template (see here):

    #!/bin/sh
    
    for HOOK in { .ScriptType }.d/*; do
        if [ -x "${HOOK}" ]; then
            /bin/sh -c $HOOK
        fi
    done
    

    This allows us to install our own hooks without modifying the hooks already installed, as well as allows the maintainers to add multiple hooks.

    Also, since git by default has a hooks directory full of examples that are not marked +x, we can safely copy them into the hooks/{SCRIPT_TYPE}.d/ directory. In your original implementation, if the original git hook is made by git and non-executable, the code won't run.

    Additionally we can chmod +x .git/hooks/{SCRIPT_TYPE}/99-rbgateway with os.Chmod.

    1. What's the significance of 00 and 99? and the .d extension?

    2. Its a convention. .d means directory, e.g. conf.d is the "configuration directory".

      the 00, 99 determine what will execute first. 00 will appear in a directory listing before 99 so 00-foo.sh will execute before 99-foo.sh using that script above.

  4. git_repository.go (Diff revision 4)
     
     
    Show all issues

    Git hooks should be named post-commit, etc. without the .sh.

  5. 
      
FL
FL
FL
jshephard
  1. 
      
  2. webhook.go (Diff revision 7)
     
     
    Show all issues

    Should the last character (',') be truncated from the error message?

    1. Leading space as well?

    2. Oops, nvm missed the message bit. Just the , perhaps.

  3. 
      
FL
FL
FL
rpolyano
  1. 
      
  2. api/api.go (Diff revision 9)
     
     
    Show all issues

    Are their constants/enums for HTTP methods somewhere?

    1. I think using a constant for this is a bit overkill :)

  3. main.go (Diff revision 9)
     
     

    Do assignments in Go not return tha value? i.e. could this be (err = watcher.Add(config.DefaultConfigPath)) != nil?

    1. What you've suggested isn't idiomatic Go.

  4. main.go (Diff revision 9)
     
     

    Interesting way to do error handling. Certainly cleaner than try/catch

  5. repositories/git_test.go (Diff revision 9)
     
     
    Show all issues

    Is there a way to not hardcode this here? Perhaps move it somewhere else in (or out of) the code?

    1. Test data should be in tests. If we put this in the filesystem, it makes tests more complicated since they now have to do IO before they can run (and IO in tests leads to slow tests over time).

  6. 
      
FL
brennie
  1. 
      
  2. api/routes.go (Diff revision 10)
     
     
    Show all issues

    Can you rewrite the summary in the imperative mood, e.g.:

    // Register a wehook.
    
  3. api/routes.go (Diff revision 10)
     
     
    Show all issues

    This should return 201 Created.

  4. api/routes.go (Diff revision 10)
     
     
    Show all issues

    Likewise here.

  5. api/routes.go (Diff revision 10)
     
     
    Show all issues

    Unnecessary, as thats the default behaviour of the HTTP standard.

  6. api/routes_test.go (Diff revision 10)
     
     
    Show all issues

    Care to put the body param before the t?

  7. config/config.go (Diff revision 10)
     
     
    Show all issues

    } on next line.

    1. Is there a preference for either, trailing , or } on the same line?

  8. config/config.go (Diff revision 10)
     
     
    Show all issues

    Is that a single space or is it a tab character? IMO we should use two (my preference) or four space indent or tab.

  9. main.go (Diff revision 10)
     
     
    Show all issues

    Missing a period.

  10. sample_config.json (Diff revision 10)
     
     
    Show all issues

    I think we should either manage webhooks exclusively via the API or exclusively via config.json. If the former, we should store them in a separate file so that adding a webhook does not cause cause the server to restart (as fs notifier currently will notice we wrote the file and).

    Or we manage them excusively via config.json, in which case a restart is to be expected.

    1. I'm inclined for the API route since its a similar workflow for github and bitbucket where we send a payload and the hosting service does its job. And based on the current implementation, if the config.json is incorrectly configured because of webhooks, it brings down the whole rb-gateway.

  11. 
      
FL
brennie
  1. 
      
  2. hooks/webhook.go (Diff revision 11)
     
     
     
     
     
    Show all issues

    We really just need the ID. Everything else we can get if we need it.

    1. We can, but this follows the same pattern as the other hosting services like github , bitcbucket, and beanstalk so I'm not sure if we should deviate from that pattern.

    2. Oh wait, unless you meant, in the post-receive.sh script itself, we don't need to pass in the message. We could just use git2go and call GetCommit(commitId) and it'll have the message?

  3. hooks/webhook.go (Diff revision 11)
     
     
     
    Show all issues

    Comments should be in proper sentences beginning with capitals and ending with periods.

    Same goes for all comments throughout.

  4. hooks/webhook.go (Diff revision 11)
     
     
    Show all issues

    Can you write your doc-comments in the imperative mood, e.g.

    "Return whether or not the webhook has valid fields."

  5. hooks/webhook.go (Diff revision 11)
     
     
    Show all issues

    remove this line.

  6. hooks/webhook.go (Diff revision 11)
     
     
     
     
    Show all issues

    Can be one statement

  7. hookscripts/post-commit.sh (Diff revision 11)
     
     
     
     
     
     
     
     
     
     
     
    Show all issues

    Ideally we can have this as a template string inside go and then replace (1) the path to rbgateway and (2) the reponame ourselves when we install it. It also would mean we don't need to include this text file when distributing rb-gateway

    1. To me it seems weird that we would ask users to install rb-gateway and change the code inside it. It seems like this is highly configurable depending on what people want to use rb-gateway for, and should be in some config file, outside of the code.

  8. hookscripts/post-commit.sh (Diff revision 11)
     
     
    Show all issues

    So this isn't strictly correct if you're not pushing to master.

    You will want to read from standard input the data about the changes received. (This links to the pre-receive hook but they get the same input).

    e.g.

    read OLD NEW REF_NAME
    
    echo "$OLD $NEW $REF_NAME" | $rbgatewaypath -send-webhook=post-commit -repository=$reponame
    

    where REF_NAME is the name of the ref being updated (e.g., master) and OLD and NEW are the new revisions.

  9. main.go (Diff revision 11)
     
     
    Show all issues

    This should live in the hooks package

  10. main.go (Diff revision 11)
     
     
    Show all issues

    Can we move the code thats executed when triggeredByHookscript is true into another function?

    That way we can do:

    if triggeredByHookscript {
        return executeHook(...)
    }
    // server stuff here
    
  11. repositories/git.go (Diff revision 11)
     
     
    Show all issues

    Testing code?

  12. repositories/git.go (Diff revision 11)
     
     
    Show all issues

    You should use "${HOOK}" here in case it contains a space.

  13. repositories/git_test.go (Diff revision 11)
     
     

    As an FYI, you can use touch filename to modify the mtime (last modified time) of filename or create filename if it doesnt exist.

  14. repositories/git_test.go (Diff revision 11)
     
     
    Show all issues

    No blank line here.

  15. 
      
FL
JT
  1. No comments, just questions. This is my first encounter with Go, but your code is very readable!

  2. api/api.go (Diff revision 12)
     
     
    Show all issues

    I've never read or worked with Go before, but I think it's pretty cool how readable some of it is. Would this syntax allow you to chain multiple methods?

    1. It's not inherently common in go (at least from what I've seen) where a lot of chaining is done but this library we use, mux takes advantage of it. But you can chain multiple methods as long as it passes back the object back. If you've used something like jquery before, it's pretty similar where a lot of the functions return some jquery object so you can chain methods.

  3. api/routes_test.go (Diff revision 12)
     
     
    Show all issues

    Another question, what does the syntax for the parameter t *testing.T mean?

    1. testing.T is golang package and it comes with a lot of built in functions. See here for more: https://golang.org/pkg/testing/

  4. 
      
JC
  1. 
      
  2. repositories/git_test.go (Diff revision 12)
     
     
    Show all issues

    I'm not familiar with go lang at all really so I'm just wondering, how come sometimes you use "if err == " and other times it's "if err = "?

    1. Not really sure what you wanted to say without the typo, but if you meant != vs == err, it's usually checking for err != nil because we want to return early (or do something early) if there's the presence of errors.

  3. 
      
FL
Review request changed
Commit:
a2de0377b0cf9d6326deb8d51440ce77bdee023c
944a59382c928842566ba088531bc6ee08b9bda9

Checks run (2 timed out)

flake8 timed out.
JSHint timed out.