Perform database evolutions using the new dependency graph.

 






 

 


  Review Request #11260
  —
  Created Nov. 2, 2020 and submitted 

 




















 

  

   Information

  





 
 

  chipx86
 









 
 

  Django Evolution
 









 
 

  master
 









 
 

  
 









 
 

  
 






 




 

  

   Reviewers

  





 
 

  django-evolution
 









 
 

  
 






 








 





  


 
 

  
Django Evolution now uses the new dependency graph to determine the

order in which evolutions and migrations will apply, allowing projects

to ensure that, for instance, certain migrations apply after certain

evolutions, or vice-versa.


Starting out, it builds up the migration plans and computes evolution

state as it did before, and then it provides this to EvolutionGraph.

To keep the same rough default ordering that we had in 2.0, the

"pre"-stage migration plan is added to the graph first, followed by

evolutions/model creations, and then the "post"-stage migration plan.

This takes advantage of the insertion-based ordering that

EvolutionGraph uses by default for any items that don't otherwise

define explicit dependencies.


It then fetches the dependency-ordered list of batches of migrations,

evolutions, and model creations that should be applied and further

processes and groups them into a new set of batches for execution.


All this work leads to a carefully-ordered set of operations that can be

performed, in the correct order, to upgrade a database no matter which

upgrade methods an app is using and in which order they need to be

applied (so long as there are evolutions hinting at the correct order).


This is the culmination of the last several weeks and 25 commits worth

of work. There's still work to add documentation, and there may be

refinements on the process, but as of now this appears to be feature

complete.

 








  


 
 

  
Unit tests pass across all versions of Python and Django, across all

databases.


Performed a new install of Review Board 4.0 (with dependency information

added) successfully on MySQL and Postgres. This wasn't possible before

outside of SQLite.


Also performed an upgrade from Review Board 3.0 to 4.0 successfully.

 







 









 





  

 


 
 

  



 

  
   


    Summary
    ID

   

  
  

   


    

     

      Perform database evolutions using the new dependency graph.
      
Django Evolution now uses the new dependency graph to determine the
order in which evolutions and migrations will apply, allowing projects
to ensure that, for instance, certain migrations apply after certain
evolutions, or vice-versa.

Starting out, it builds up the migration plans and computes evolution
state as it did before, and then it provides this to `EvolutionGraph`.
To keep the same rough default ordering that we had in 2.0, the
"pre"-stage migration plan is added to the graph first, followed by
evolutions/model creations, and then the "post"-stage migration plan.
This takes advantage of the insertion-based ordering that
`EvolutionGraph` uses by default for any items that don't otherwise
define explicit dependencies.

It then fetches the dependency-ordered list of batches of migrations,
evolutions, and model creations that should be applied and further
processes and groups them into a new set of batches for execution.

All this work leads to a carefully-ordered set of operations that can be
performed, in the correct order, to upgrade a database no matter which
upgrade methods an app is using and in which order they need to be
applied (so long as there are evolutions hinting at the correct order).

This is the culmination of the last several weeks and 25 commits worth
of work. There's still work to add documentation, and there may be
refinements on the process, but as of now this appears to be feature
complete.

     


    		
    e2042342c4019b33711268172d1f16dad003a7da

   



  
 




 













 






 
 
 

  

   

    

     
    

    
    
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
    

   

   

    
     

      Description
      From
      Last Updated
     

    
    



     

      
       
       
This method is absolutely massive. Can we maybe pull out some of the steps into their own methods?

      		
      
       daviddavid
      
      
       
      
     




     

      
       
       
E501 line too long (80 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (88 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (88 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (94 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (98 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (80 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     




     

      
       
       
E501 line too long (88 > 79 characters)

      		
      
       reviewbotreviewbot
      
      
       
      
     


    
   

  

 







 






  

 



















 




 

  


   

   

    
 
Checks run (1 failed, 1 succeeded)


    
   


  

  

  





 

 flake8
 failed.
 





 

 JSHint
 passed.
 













 
flake8

 


 










  

 





 














 





 

  

   




david




   

   







  

 



 

  


   

   

    
 
david


    
   


  

  

  



    




  1. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 

















  2. 
 
    
  
  
    
  
 
    
  
    
   
    
    
     
    
      
       django_evolution/evolve.py
       


        (Diff revision 1)


       
      
     
    
        
    
     
     
    

     
     
    

        
   
    
  
    
 
    

   
    
    
    




    
 
    
  
  
  

  
  
   Show all issues
  
 
    

    



    
    This method is absolutely massive. Can we maybe pull out some of the steps into their own methods?
    
  
    
 
    

 
    
  


    
 
    
 
      

  


    1. 
 
 
      
 
      
  
 
      
 
      
  
      Yeahh I was considering that. Let me toy with it.
      
 
      

      2. 


 
    


    

 
    

    3. 




  3. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    4. 




  

 
















 



 

  

   




chipx86




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  









  

   

    Change Summary:
   

   

    
    

  • Split up prepare_tasks() into multiple independent methods, to keep things maintainable.
    • 

  • Stopped storing computed evolver-wide EvolveAppTask state on the class, and instead stored it in a dictionary in the evolver instance.
    • 


   

  








  

   

    Commits:
   

   

    





 

  
   


    

    Summary
    ID

   

  
  

   


    

    

     

      Perform database evolutions using the new dependency graph.
      
Django Evolution now uses the new dependency graph to determine the
order in which evolutions and migrations will apply, allowing projects
to ensure that, for instance, certain migrations apply after certain
evolutions, or vice-versa.

Starting out, it builds up the migration plans and computes evolution
state as it did before, and then it provides this to `EvolutionGraph`.
To keep the same rough default ordering that we had in 2.0, the
"pre"-stage migration plan is added to the graph first, followed by
evolutions/model creations, and then the "post"-stage migration plan.
This takes advantage of the insertion-based ordering that
`EvolutionGraph` uses by default for any items that don't otherwise
define explicit dependencies.

It then fetches the dependency-ordered list of batches of migrations,
evolutions, and model creations that should be applied and further
processes and groups them into a new set of batches for execution.

All this work leads to a carefully-ordered set of operations that can be
performed, in the correct order, to upgrade a database no matter which
upgrade methods an app is using and in which order they need to be
applied (so long as there are evolutions hinting at the correct order).

This is the culmination of the last several weeks and 25 commits worth
of work. There's still work to add documentation, and there may be
refinements on the process, but as of now this appears to be feature
complete.

     


    		
    555a17a614eef84d3653b6bcc17bb80d970d3e27

   




   


    

    

     

      Perform database evolutions using the new dependency graph.
      
Django Evolution now uses the new dependency graph to determine the
order in which evolutions and migrations will apply, allowing projects
to ensure that, for instance, certain migrations apply after certain
evolutions, or vice-versa.

Starting out, it builds up the migration plans and computes evolution
state as it did before, and then it provides this to `EvolutionGraph`.
To keep the same rough default ordering that we had in 2.0, the
"pre"-stage migration plan is added to the graph first, followed by
evolutions/model creations, and then the "post"-stage migration plan.
This takes advantage of the insertion-based ordering that
`EvolutionGraph` uses by default for any items that don't otherwise
define explicit dependencies.

It then fetches the dependency-ordered list of batches of migrations,
evolutions, and model creations that should be applied and further
processes and groups them into a new set of batches for execution.

All this work leads to a carefully-ordered set of operations that can be
performed, in the correct order, to upgrade a database no matter which
upgrade methods an app is using and in which order they need to be
applied (so long as there are evolutions hinting at the correct order).

This is the culmination of the last several weeks and 25 commits worth
of work. There's still work to add documentation, and there may be
refinements on the process, but as of now this appears to be feature
complete.

     


    		
    e2042342c4019b33711268172d1f16dad003a7da

   



  
 







   

  





  

   

    Diff:
   

   

    
 Revision 2 (+4398 -508)
Show changes
 
    django_evolution/evolve.py 
    django_evolution/mutations.py 
    django_evolution/mutators.py 
    django_evolution/tests/base_test_case.py 
    django_evolution/tests/test_evolver.py 
    django_evolution/tests/utils.py 
    django_evolution/tests/db/mysql.py 
    django_evolution/tests/db/postgresql.py 
  2 more
 

   

  











 

  
Checks run (1 failed, 1 succeeded)



  

 flake8
 failed.
 





  

 JSHint
 passed.
 





 







 
flake8

 


 















  

 
















 





 

  

   




david




   

   







  

 



 

  


   

   

    
 
david


    
   


  

  

  



    




  1. 
 
    
  
    
   
    Looks great now!
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 















  2. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 



 

  

   




chipx86




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  







  

   

    Status:
   

   


    Completed

   

  




  

   

    Change Summary:
   

   

    
Pushed to master (182306d)