• 
  
    
 
    
 
    

    



    
    
    
    
     
    




    
 
    

  
    



    
 

 

  

  

 

    


  
    





    
 
    



  

    
 
 
    
  
    Update CredentialChecker for new tool support and parsing improvements.
    
 
    

    



 
    
 
    

  Review Request #11579
  —
  Created April 6, 2021 and submitted 

 
    


    




    




    






 
    
  
    
   Information

  
    



    
 
 
    
  chipx86
 
    

    





    
 
 
    
  ReviewBot
 
    

    





    
 
 
    
  release-3.0.x
 
    

    





    
 
 
    
  
 
    

    





    
 
 
    
  
 
    

    



 
    



 
    
  
    
   Reviewers

  
    



    
 
 
    
  reviewbot
 
    

    





    
 
 
    
  
 
    

    



 
    



    


    
 
    




  

    
 
 
    
  
    This updates CredentialChckerTool to subclass the newer BaseTool.
    

    The original implementation had some regex patterns that didn't survive
    
the later stages of development, when combining into a single verbose
    
regex. Any pattern that expected a space (such as PGP private key
    
blocks) failed to match. This is now fixed.
    

    The GitHub token checks have been greatly improved. As of April 2021,
    
GitHub introduced a new token format that can more easily be scanned,
    
and contains a checksum as the final 6 characters of the token, allowing
    
a scanner to detect real vs. fake tokens (though not active vs.
    
inactive). We now support enhanced validation for these GitHub tokens,
    
and have logic in place to allow for enhanced validation for future
    
tokens as well.
    

    Most previous token types have been updated for modern token formats and
    
to reduce false-positives by preventing the tokens from being embedded
    
in other strings.
    

    Token support has been added for:
    

      

    • Asana
      • 

    • Discord
      • 

    • Dropbox
      • 

    • Google Cloud Platform
      • 

    • JSON Web Tokens
      • 

    • Mailchimp
      • 

    • Mailgun
      • 

    • NPM
      • 

    • PyPI
      • 

    • Slack (WebHooks)
      • 

    • SSL Certificates
      • 

    • Stripe
      • 

    • Twilio
      • 

    
 
    

    





  

    
 
 
    
  
    Unit tests pass on Python 2.7 and 3.x.
    

    Extensively tested the new GitHub token validation logic. I've tested this
    
with a number of real-world tokens, along with garbage tokens, ensuring that
    
the real ones always validated and matched.
    
 
    

    




 
    

    





    
 
    




  
    
 
 
    
  


    
 
    
  
   
    

    Summary
    ID

   
    
      
  

   
    

    

     
    
      Update CredentialChecker for new tool support and parsing improvements.
      
    This updates `CredentialChckerTool` to subclass the newer `BaseTool`.

The original implementation had some regex patterns that didn't survive
the later stages of development, when combining into a single verbose
regex. Any pattern that expected a space (such as PGP private key
blocks) failed to match. This is now fixed.

The GitHub token checks have been greatly improved. As of April 2021,
GitHub introduced a new token format that can more easily be scanned,
and contains a checksum as the final 6 characters of the token, allowing
a scanner to detect real vs. fake tokens (though not active vs.
inactive). We now support enhanced validation for these GitHub tokens,
and have logic in place to allow for enhanced validation for future
tokens as well.

Most previous token types have been updated for modern token formats and
to reduce false-positives by preventing the tokens from being embedded
in other strings.

New token have been added for:

* Asana
* Discord
* Dropbox
* Google Cloud Platform
* JSON Web Tokens
* Mailchimp
* Mailgun
* NPM
* PyPI
* Slack (WebHooks)
* SSL Certificates
* Stripe
* Twilio
    
     
    

        		
    95d6da275c2637656ff80ceb6fb41d78f220b8b2

   
    


      
 
    

    
    
 
    

    






 





    
 
 
 
    
  
    
   
    
    
    
     
    
    
    
      
     
    • 
      
     
      • 
     
    • 
      
     
      • 
     
    • 
      
     
      • 
     
    • 
      
     
      • 
     
    • 
      
     
      • 
    
    
   
    
   
    
    
     
    
      Description
      From
      Last Updated
     
    
        
    



     
    
      
       
       
    E501 line too long (93 > 79 characters)
    
          		
      
       reviewbotreviewbot
      
      
       
      
     
    



     
    
      
       
       
    E501 line too long (80 > 79 characters)
    
          		
      
       reviewbotreviewbot
      
      
       
      
     
    

        
   
    
  
    
 
    

    




 
    

    



  
    
 
    

    


    












    
 




 

    


 













    
 





 
    
  
    
   
    



david




   
    
   
    





    
  
    
 
    


 
    
  
    

   
    
   
    
    
     
david


    
    
   
    

  
    
  
    
  
    


      




    1. 
 
      
  
      
   
      Ship It!
      
  
      
 
      

 
      
  


      
 
      
 
        

  

 
      


      

 
      

      2. 















    2. 
 
      
  
      
   
      
  
      
 
      

 
      
  


      
 
      
 
        

  

 
      


      

 
      

      3. 


    

  
    
 
    

    












    
 



 
    
  
    
   
    



chipx86




   
    
   
    
  
    
 
    


 
    
  
    

   
    
   
    
    
     Review request changed
    
    
   
    

  
    
  
    
  
    




    



  
    
   
    
    Change Summary:
   
    
   
    
    
      

    • Fixed up many of the previously-implemented tokens to reduce false-positives and match current token formats for services.
      • 

    • Grouped tokens by sections (Vendor-identifiable, vendor-likely, and general patterns), for priority matching.
      • 

    • Added "as of" dates for the tokens, to ease future maintenance.
      • 

    • Added a blurb to the comment stating that the user may want to revoke the existing token.
      • 

    • Added a bunch of new supported token types.
      • 

    
   
    
  
    







  
    
   
    
    Description:
   
    
   
    
    
         
    This updates CredentialChckerTool to subclass the newer BaseTool.
         

         
    The original implementation had some regex patterns that didn't survive
         the later stages of development, when combining into a single verbose
         regex. Any pattern that expected a space (such as PGP private key
         blocks) failed to match. This is now fixed.
         

         
    The GitHub token checks have been greatly improved. As of April 2021,
         GitHub introduced a new token format that can more easily be scanned,
         and contains a checksum as the final 6 characters of the token, allowing
         a scanner to detect real vs. fake tokens (though not active vs.
         inactive). We now support enhanced validation for these GitHub tokens,
         and have logic in place to allow for enhanced validation for future
         tokens as well.
       + 

       + 
    Most previous token types have been updated for modern token formats and
       + to reduce false-positives by preventing the tokens from being embedded
       + in other strings.
       + 

       + 
    Token support has been added for:
       + 

       + 
    • Asana
       + 
    • Discord
       + 
    • Dropbox
       + 
    • Google Cloud Platform
       + 
    • JSON Web Tokens
       + 
    • Mailchimp
       + 
    • Mailgun
       + 
    • NPM
       + 
    • PyPI
       + 
    • Slack (WebHooks)
       + 
    • SSL Certificates
       + 
    • Stripe
       + 
    • Twilio
    
   
    
  
    




  
    
   
    
    Commits:
   
    
   
    
    
    



    
 
    
  
   
    

    

    Summary
    ID

   
    
      
  

   
    

    

    

     
    
      Update CredentialChecker for new tool support and parsing improvements.
      
    This updates `CredentialChckerTool` to subclass the newer `BaseTool`.

The original implementation had some regex patterns that didn't survive
the later stages of development, when combining into a single verbose
regex. Any pattern that expected a space (such as PGP private key
blocks) failed to match. This is now fixed.

The GitHub token checks have been greatly improved. As of April 2021,
GitHub introduced a new token format that can more easily be scanned,
and contains a checksum as the final 6 characters of the token, allowing
a scanner to detect real vs. fake tokens (though not active vs.
inactive). We now support enhanced validation for these GitHub tokens,
and have logic in place to allow for enhanced validation for future
tokens as well.
    
     
    

        		
    f21c19da7d3af256ce67f8f48f53a38fd68b90b8

   
    



   
    

    

    

     
    
      Update CredentialChecker for new tool support and parsing improvements.
      
    This updates `CredentialChckerTool` to subclass the newer `BaseTool`.

The original implementation had some regex patterns that didn't survive
the later stages of development, when combining into a single verbose
regex. Any pattern that expected a space (such as PGP private key
blocks) failed to match. This is now fixed.

The GitHub token checks have been greatly improved. As of April 2021,
GitHub introduced a new token format that can more easily be scanned,
and contains a checksum as the final 6 characters of the token, allowing
a scanner to detect real vs. fake tokens (though not active vs.
inactive). We now support enhanced validation for these GitHub tokens,
and have logic in place to allow for enhanced validation for future
tokens as well.

Most previous token types have been updated for modern token formats and
to reduce false-positives by preventing the tokens from being embedded
in other strings.

New token have been added for:

* Asana
* Discord
* Dropbox
* Google Cloud Platform
* JSON Web Tokens
* Mailchimp
* Mailgun
* NPM
* PyPI
* Slack (WebHooks)
* SSL Certificates
* Stripe
* Twilio
    
     
    

        		
    95d6da275c2637656ff80ceb6fb41d78f220b8b2

   
    


      
 
    

    


    

   
    
  
    




  
    
   
    
    Diff:
   
    
   
    
    
     Revision 2 (+1954 -80)
    Show changes
     
        bot/reviewbot/tools/rbcredentialchecker.py 
        bot/reviewbot/tools/testing/testcases.py 
        bot/reviewbot/tools/tests/test_rbcredentialchecker.py 
        bot/reviewbot/utils/text.py 
        bot/reviewbot/utils/tests/test_text.py 
     
    
   
    
  
    





    



    
 
    
  
    Checks run (1 failed, 1 succeeded)
    


  
    
 flake8
 failed.
 


    


  
    
 JSHint
 passed.
 


    


 
    





    
 
    flake8
    
 
    

 

    








    




  
    
 
    

    












    
 



 
    
  
    
   
    



chipx86




   
    
   
    
  
    
 
    


 
    
  
    

   
    
   
    
    
     Review request changed
    
    
   
    

  
    
  
    
  
    




    

  
    
   
    
    Status:
   
    
   
    

    Completed

   
    
  
    



  
    
   
    
    Change Summary:
   
    
   
    
    
    Pushed to release-3.0.x (ea1f4a7)