Fix a test ordering and state leak issues affecting tool/logging tests.

 






 

 


  Review Request #12399
  —
  Created June 21, 2022 and submitted 

 




















 

  

   Information

  





 
 

  chipx86
 









 
 

  Review Board
 









 
 

  release-5.0.x
 









 
 

  
 









 
 

  
 






 




 

  

   Reviewers

  





 
 

  reviewboard
 









 
 

  
 






 








 





  


 
 

  
Depending on the order in which tests were run, the new Tool conflict

tests in test_registry would fail. One failure had to do with the

presence or lack of Tool objects in the database, and the other had to

do with an assertLogs() failing to catch logged warnings.


There were three issues behind these. One was a failure in the test, and

two were state consistency issues:


    

  1. 

    The new unit test needed Tool objects but failed to use the proper
    
   fixture. This wasn't caught initially due to the next issue.
    

    2. 

  2. 

    Tool objects were sometimes populated when not needed by a unit
    
   test, and could be populated at different levels of test setup,
    
   causing state inconsistency depending on which tests were run first.
    
   The solution to this is to avoid automatically populating Tool
    
   objects in SCMToolRegistry when running unit tests, allowing the
    
   previous fixture behavior to work as it did before, and to populate
    
   at the right times.
    

    3. 

  3. 

    A unit test in test_review.py globally disabled warnings. I can't
    
   determine why it did this, but it was likely for very old historical
    
   reasons. This affected subsequent tests. This code has now been
    
   removed.
    

    4. 


 








  


 
 

  
Tested full test runs and various combinations of suites. All the ones

that failed before pass now.

 







 









 





  

 


 
 

  



 

  
   


    Summary
    ID

   

  
  

   


    

     

      Fix a test ordering and state leak issues affecting tool/logging tests.
      
Depending on the order in which tests were run, the new `Tool` conflict
tests in `test_registry` would fail. One failure had to do with the
presence or lack of `Tool` objects in the database, and the other had to
do with an `assertLogs()` failing to catch logged warnings.

There were three issues behind these. One was a failure in the test, and
two were state consistency issues:

1. The new unit test needed `Tool` objects but failed to use the proper
   fixture. This wasn't caught initially due to the next issue.

2. `Tool` objects were sometimes populated when not needed by a unit
   test, and could be populated at different levels of test setup,
   causing state inconsistency depending on which tests were run first.
   The solution to this is to avoid automatically populating `Tool`
   objects in `SCMToolRegistry` when running unit tests, allowing the
   previous fixture behavior to work as it did before, and to populate
   at the right times.

3. A unit test in `test_review.py` globally disabled warnings. I can't
   determine why it did this, but it was likely for very old historical
   reasons. This affected subsequent tests. This code has now been
   removed.

     


    		
    667a7674f949867e636a9e26a01c5a2ddc3762ef

   



  
 




 













 






 
 
 

  

   

    

     
    

    
    
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
    

   

   

    
     

      Description
      From
      Last Updated
     

    
    



     

      
       
       
So the use of @add_fixtures(['test_scmtools']) and fixtures = ['test_scmtools'] work as they did before? Does this mean they are no …

      		
      
       maubinmaubin
      
      
       
      
     


    
   

  

 







 






  

 



















 




 




 














 





 

  

   




david




   

   







  

 



 

  


   

   

    
 
david



    
   


  

  

  



    




  1. 
 
    
  
    
   
    Ship It!
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 















  2. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 





 

  

   




maubin




   

   







  

 



 

  


   

   

    
 
maubin



    
   


  

  

  



    




  1. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 







  2. 
 
    
  
  
    
  
   
    
    
    




    
 
    
  
  
  

  
  
   Show all issues
  
 
    

    



    
    So the use of @add_fixtures(['test_scmtools']) and fixtures = ['test_scmtools'] work as they did before? Does this mean they are no longer deprecated and that they need to be present to use methods like create_repository
    
  
    
 
    

 
    
  


    
 
    
 
      

  


    1. 
 
 
      
 
      
  
 
      
 
      
  
      Usage of test_scmtools will ensure that the database contains Tools, so we'll still need to use them for any tests that in any way depend on a Tool. We should require this less and less, as we rely more on SCMTool IDs. This particular test is all about Tool-based conflicts, so it needed it, but it was missed before due to the state leakage.
      
 
      

      2. 


 
    


    

 
    

    3. 













  3. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    4. 




  

 
















 



 

  

   




chipx86




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  







  

   

    Status:
   

   


    Completed

   

  




  

   

    Change Summary:
   

   

    
Pushed to release-5.0.x (9f51ffe)