Add full-repository support for Review Bot tools with git.

Review Request #8817 - Created March 15, 2017 and submitted

David Trowbridge
ReviewBot
master
678a0a2...
reviewbot

This change adds a way for Review Bot tools to operate with an entire checkout
of a (git) repository. This requires a fair amount of configuration on the
worker side, which I'm planning to document more thoroughly in a forthcoming
user manual.

When one or more repositories is configured, the worker will listen on a new
set of queues (the product of the list of repositories and the available tools
which can run on full repos). This way, a worker will only pick up tasks for
repositories which it has available.

When a change using a repository is triggered for review, the repository is
either cloned (if it doesn't exist), or fetched. This repo is then
shallow-cloned to the desired parent revision and the patched files are
downloaded. The tool can then do whatever it needs with the repo and post its
review.

  • Verified that clone/fetch and the shallow cloned worked as expected.
  • Used this in conjunction with a Clang Static Analyzer tool.
  • 0
  • 0
  • 12
  • 2
  • 14
Description From Last Updated
Review Bot
  1. Tool: Pyflakes
    Processed Files:
        bot/reviewbot/tasks.py
        bot/reviewbot/tools/__init__.py
        bot/reviewbot/repositories.py
        extension/reviewbotext/integration.py
        bot/reviewbot/utils/filesystem.py
        bot/reviewbot/tools/buildbot.py
        bot/reviewbot/celery.py
        extension/reviewbotext/resources.py
        extension/reviewbotext/models.py
        bot/reviewbot/config.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        bot/reviewbot/tasks.py
        bot/reviewbot/tools/__init__.py
        bot/reviewbot/repositories.py
        extension/reviewbotext/integration.py
        bot/reviewbot/utils/filesystem.py
        bot/reviewbot/tools/buildbot.py
        bot/reviewbot/celery.py
        extension/reviewbotext/resources.py
        extension/reviewbotext/models.py
        bot/reviewbot/config.py
    
    
  2. bot/reviewbot/celery.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (81 > 79 characters)
    
  3. bot/reviewbot/repositories.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (80 > 79 characters)
    
  4. bot/reviewbot/repositories.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (85 > 79 characters)
    
  5. bot/reviewbot/tasks.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (81 > 79 characters)
    
  6. bot/reviewbot/tasks.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (83 > 79 characters)
    
  7. bot/reviewbot/tools/__init__.py (Diff revision 1)
     
     
     'os' imported but unused
    
  8. bot/reviewbot/tools/buildbot.py (Diff revision 1)
     
     
    Col: 80
     E501 line too long (81 > 79 characters)
    
  9. bot/reviewbot/utils/filesystem.py (Diff revision 1)
     
     
     undefined name 'logging'
    
  10. bot/reviewbot/utils/filesystem.py (Diff revision 1)
     
     
     undefined name 'logging'
    
  11. 
      
David Trowbridge
Review Bot
  1. Tool: PEP8 Style Checker
    Processed Files:
        bot/reviewbot/tasks.py
        bot/reviewbot/tools/__init__.py
        bot/reviewbot/repositories.py
        extension/reviewbotext/integration.py
        bot/reviewbot/utils/filesystem.py
        bot/reviewbot/tools/buildbot.py
        bot/reviewbot/celery.py
        extension/reviewbotext/resources.py
        extension/reviewbotext/models.py
        bot/reviewbot/config.py
    
    
    
    Tool: Pyflakes
    Processed Files:
        bot/reviewbot/tasks.py
        bot/reviewbot/tools/__init__.py
        bot/reviewbot/repositories.py
        extension/reviewbotext/integration.py
        bot/reviewbot/utils/filesystem.py
        bot/reviewbot/tools/buildbot.py
        bot/reviewbot/celery.py
        extension/reviewbotext/resources.py
        extension/reviewbotext/models.py
        bot/reviewbot/config.py
    
    
  2. 
      
Barret Rennie
  1. 
      
  2. bot/reviewbot/celery.py (Diff revision 2)
     
     

    You don't need .keys() to iterate over the dict's keys. Iterating over it will yield the keys.

  3. 
      
Christian Hammond
  1. 
      
  2. bot/reviewbot/repositories.py (Diff revision 2)
     
     
     
     
     
     
     
     
     
     

    This is going to get big over time as we add support for more types. Let's split this up now, have a directory for repositories and a module for the base, another for git.

    1. I think I'd prefer to split it up when it actually gets big. This whole file is just 102 lines long.

  3. bot/reviewbot/tasks.py (Diff revision 2)
     
     
  4. extension/reviewbotext/integration.py (Diff revision 2)
     
     
     
     
     
     
     

    Is there any mechanism to send parameters using something akin to keyword arguments? Maybe send a dictionary across? I worry about expanding this down the road and introducing compatibility nightmares.

    1. Hmm. I'm not sure. I'd really rather not deal with that now, though. I'll make a note to investigate.

  5. 
      
David Trowbridge
Barret Rennie
  1. 
      
  2. bot/reviewbot/celery.py (Diff revision 3)
     
     

    Unnecessary blank line.

  3. 
      
David Trowbridge
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to master (32d991b)
Loading...