Apply migrations during the evolution process.

Review Request #10577 — Created June 4, 2019 and submitted

Information

Django Evolution
master

Reviewers

Evolver has been updated to factor in migrations during the evolution
process. Now, whenever an app is scheduled for any upgrade, its
migrations and evolutions will both be considered and applied.

The way it work is that, when first beginning a schema upgrade process
for an app, EvolveAppTask will set up state for a potential
migrations, which would occur after any evolutions have completed. It
considers only apps that use migrations instead of evolutions, or apps
that move from evolutions to migrations by way of the
MoveToDjangoMigrations mutation. A migration plan is built around
this, and the migrations that would be applied are calculated and
recorded.

After determining the migrations that would be applied, the normal
evolution preparation process begins as normal, determining any
evolutions that need to be applied.

Once this has been calculated for all pending apps, the execution phase
begins. This starts by creating any non-migration-managed tables and
applying any evolutions, as before. Once this has completed for all
apps, the migration plan is executed, migrating all apps that were
either migration-only or have applied a MoveToDjangoMigrations
mutation. The signatures are then updated appropriately, reflecting the
new state.

This is all done seamlessly through a single Evolver instance. No need
to call out to the migrate command or anything.

There are a couple of limitations right now:

  1. Apps using migrations cannot go back to evolutions. This is probably
    going to be fine in practice.

  2. Models managed by migrations that are newly-created cannot reference
    models managed by evolutions. Migration-backed models are always
    created by other models. In practice, this is also probably fine.
    Solving this would result in a more complex process involving a
    dependency graph between evolutions and migrations, and a way for
    either to mark dependencies on the other, which adds a lot of
    complexity.

Unit tests pass on all supported versions of Django.

Manually tested upgrading a Review Board database made on 3.0 (with
Django 1.6) to 4.0 (Django 1.11). The process involves a mix of
migrations and evolutions.

Also tested creating a new database from scratch on Django 1.11,
using a mixture of migrations and evolutions.

Summary ID
Apply migrations during the evolution process.
`Evolver` has been updated to factor in migrations during the evolution process. Now, whenever an app is scheduled for any upgrade, its migrations and evolutions will both be considered and applied. The way it work is that, when first beginning a schema upgrade process for an app, `EvolveAppTask` will set up state for a potential migrations, which would occur after any evolutions have completed. It considers only apps that use migrations instead of evolutions, or apps that move from evolutions to migrations by way of the `MoveToDjangoMigrations` mutation. A migration plan is built around this, and the migrations that would be applied are calculated and recorded. After determining the migrations that would be applied, the normal evolution preparation process begins as normal, determining any evolutions that need to be applied. Once this has been calculated for all pending apps, the execution phase begins. This starts by creating any non-migration-managed tables and applying any evolutions, as before. Once this has completed for all apps, the migration plan is executed, migrating all apps that were either migration-only or have applied a `MoveToDjangoMigrations` mutation. The signatures are then updated appropriately, reflecting the new state. This is all done seamlessly through a single `Evolver` instance. No need to call out to the `migrate` command or anything. There are a couple of limitations right now: 1. Apps using migrations cannot go back to evolutions. This is probably going to be fine in practice. 2. Models managed by migrations that are newly-created cannot reference models managed by evolutions. Migration-backed models are always created by other models. In practice, this is also probably fine. Solving this would result in a more complex process involving a dependency graph between evolutions and migrations, and a way for either to mark dependencies on the other, which adds a lot of complexity.
772791cb23443a74721c460c8bc7d350d470720b
Description From Last Updated

E131 continuation line unaligned for hanging indent

reviewbotreviewbot
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

  • django_evolution/evolve.py (Diff revision 1)
     
     
    Show all issues
    E131 continuation line unaligned for hanging indent
    1. For the record, this is how pycodestyle has you fix this warning:

      excluded_targets = set(
          (applied_app_label, applied_migration_name)
          for applied_app_label, applied_migration_names in
          six.iteritems(applied_migrations)
          for applied_migration_name in applied_migration_names
      )
      

      No thanks.

  • 
      
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to master (8f8ec3c)