• 
      

    Perform database evolutions using the new dependency graph.

    Review Request #11260 — Created Nov. 2, 2020 and submitted

    Information

    Django Evolution
    master

    Reviewers

    Django Evolution now uses the new dependency graph to determine the
    order in which evolutions and migrations will apply, allowing projects
    to ensure that, for instance, certain migrations apply after certain
    evolutions, or vice-versa.

    Starting out, it builds up the migration plans and computes evolution
    state as it did before, and then it provides this to EvolutionGraph.
    To keep the same rough default ordering that we had in 2.0, the
    "pre"-stage migration plan is added to the graph first, followed by
    evolutions/model creations, and then the "post"-stage migration plan.
    This takes advantage of the insertion-based ordering that
    EvolutionGraph uses by default for any items that don't otherwise
    define explicit dependencies.

    It then fetches the dependency-ordered list of batches of migrations,
    evolutions, and model creations that should be applied and further
    processes and groups them into a new set of batches for execution.

    All this work leads to a carefully-ordered set of operations that can be
    performed, in the correct order, to upgrade a database no matter which
    upgrade methods an app is using and in which order they need to be
    applied (so long as there are evolutions hinting at the correct order).

    This is the culmination of the last several weeks and 25 commits worth
    of work. There's still work to add documentation, and there may be
    refinements on the process, but as of now this appears to be feature
    complete.

    Unit tests pass across all versions of Python and Django, across all
    databases.

    Performed a new install of Review Board 4.0 (with dependency information
    added) successfully on MySQL and Postgres. This wasn't possible before
    outside of SQLite.

    Also performed an upgrade from Review Board 3.0 to 4.0 successfully.

    Summary ID
    Perform database evolutions using the new dependency graph.
    Django Evolution now uses the new dependency graph to determine the order in which evolutions and migrations will apply, allowing projects to ensure that, for instance, certain migrations apply after certain evolutions, or vice-versa. Starting out, it builds up the migration plans and computes evolution state as it did before, and then it provides this to `EvolutionGraph`. To keep the same rough default ordering that we had in 2.0, the "pre"-stage migration plan is added to the graph first, followed by evolutions/model creations, and then the "post"-stage migration plan. This takes advantage of the insertion-based ordering that `EvolutionGraph` uses by default for any items that don't otherwise define explicit dependencies. It then fetches the dependency-ordered list of batches of migrations, evolutions, and model creations that should be applied and further processes and groups them into a new set of batches for execution. All this work leads to a carefully-ordered set of operations that can be performed, in the correct order, to upgrade a database no matter which upgrade methods an app is using and in which order they need to be applied (so long as there are evolutions hinting at the correct order). This is the culmination of the last several weeks and 25 commits worth of work. There's still work to add documentation, and there may be refinements on the process, but as of now this appears to be feature complete.
    e2042342c4019b33711268172d1f16dad003a7da
    Description From Last Updated

    This method is absolutely massive. Can we maybe pull out some of the steps into their own methods?

    daviddavid

    E501 line too long (80 > 79 characters)

    reviewbotreviewbot

    E501 line too long (88 > 79 characters)

    reviewbotreviewbot

    E501 line too long (88 > 79 characters)

    reviewbotreviewbot

    E501 line too long (94 > 79 characters)

    reviewbotreviewbot

    E501 line too long (98 > 79 characters)

    reviewbotreviewbot

    E501 line too long (80 > 79 characters)

    reviewbotreviewbot

    E501 line too long (88 > 79 characters)

    reviewbotreviewbot
    Checks run (1 failed, 1 succeeded)
    flake8 failed.
    JSHint passed.

    flake8

    david
    1. 
        
    2. django_evolution/evolve.py (Diff revision 1)
       
       
      Show all issues

      This method is absolutely massive. Can we maybe pull out some of the steps into their own methods?

      1. Yeahh I was considering that. Let me toy with it.

    3. 
        
    chipx86
    Review request changed
    Change Summary:
    • Split up prepare_tasks() into multiple independent methods, to keep things maintainable.
    • Stopped storing computed evolver-wide EvolveAppTask state on the class, and instead stored it in a dictionary in the evolver instance.
    Commits:
    Summary ID
    Perform database evolutions using the new dependency graph.
    Django Evolution now uses the new dependency graph to determine the order in which evolutions and migrations will apply, allowing projects to ensure that, for instance, certain migrations apply after certain evolutions, or vice-versa. Starting out, it builds up the migration plans and computes evolution state as it did before, and then it provides this to `EvolutionGraph`. To keep the same rough default ordering that we had in 2.0, the "pre"-stage migration plan is added to the graph first, followed by evolutions/model creations, and then the "post"-stage migration plan. This takes advantage of the insertion-based ordering that `EvolutionGraph` uses by default for any items that don't otherwise define explicit dependencies. It then fetches the dependency-ordered list of batches of migrations, evolutions, and model creations that should be applied and further processes and groups them into a new set of batches for execution. All this work leads to a carefully-ordered set of operations that can be performed, in the correct order, to upgrade a database no matter which upgrade methods an app is using and in which order they need to be applied (so long as there are evolutions hinting at the correct order). This is the culmination of the last several weeks and 25 commits worth of work. There's still work to add documentation, and there may be refinements on the process, but as of now this appears to be feature complete.
    555a17a614eef84d3653b6bcc17bb80d970d3e27
    Perform database evolutions using the new dependency graph.
    Django Evolution now uses the new dependency graph to determine the order in which evolutions and migrations will apply, allowing projects to ensure that, for instance, certain migrations apply after certain evolutions, or vice-versa. Starting out, it builds up the migration plans and computes evolution state as it did before, and then it provides this to `EvolutionGraph`. To keep the same rough default ordering that we had in 2.0, the "pre"-stage migration plan is added to the graph first, followed by evolutions/model creations, and then the "post"-stage migration plan. This takes advantage of the insertion-based ordering that `EvolutionGraph` uses by default for any items that don't otherwise define explicit dependencies. It then fetches the dependency-ordered list of batches of migrations, evolutions, and model creations that should be applied and further processes and groups them into a new set of batches for execution. All this work leads to a carefully-ordered set of operations that can be performed, in the correct order, to upgrade a database no matter which upgrade methods an app is using and in which order they need to be applied (so long as there are evolutions hinting at the correct order). This is the culmination of the last several weeks and 25 commits worth of work. There's still work to add documentation, and there may be refinements on the process, but as of now this appears to be feature complete.
    e2042342c4019b33711268172d1f16dad003a7da

    Checks run (1 failed, 1 succeeded)

    flake8 failed.
    JSHint passed.

    flake8

    david
    1. Looks great now!

    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to master (182306d)