• 
      

    Add support for just-in-time SQL code generation for databases.

    Review Request #11089 — Created July 20, 2020 and submitted

    Information

    Django Evolution
    master

    Reviewers

    The way Django Evolution worked before was that it'd compute a set of
    operations that needed to be performed to get the database into the
    state needed for the evolutions, and it'd ask the database evolution
    backends to generate suitable SQL for later execution.

    This generally works, but there are cases in which the SQL needs to be a
    bit more dynamic than that, making use of state that doesn't exist until
    it comes time to execute that SQL statement. This will specifically be
    required for SQLite primary key modifications in modern versions of
    SQLite and Django.

    Now, rather than just returning hard-coded SQL statements, an
    SQLResult's list of statements can now contain a callback function
    that takes a cursor and returns new statements. This will be called when
    it'd otherwise be time to execute the SQL statement for that entry in
    the list.

    To make this happen, some part of the execution/logging code had to
    change. Previously, we'd normalize the list of SQL statements for
    output, and then normalize a different way for execution (the main
    differences between that we'd apply parameters to format strings for
    logging, but pass them separately to the database backend for
    execution). Since we need the callbacks to execute only once, these
    operations had to merge.

    To do this, there's now a private function dedicated to taking a list of
    SQL statements/callbacks and yielding results one-by-one. There's then a
    function, run_sql(), which provides the main logic for both execution
    and capturing for logging, allowing both to happen in one pass. The
    original execute_sql() and write_sql() methods wrap this, behaving
    as before but, in the case of execute_sql(), allowing the caller to
    capture as part of the standard execution process.

    Unit tests pass for all supported versions of Django and all databases.

    Summary ID
    Add support for just-in-time SQL code generation for databases.
    The way Django Evolution worked before was that it'd compute a set of operations that needed to be performed to get the database into the state needed for the evolutions, and it'd ask the database evolution backends to generate suitable SQL for later execution. This generally works, but there are cases in which the SQL needs to be a bit more dynamic than that, making use of state that doesn't exist until it comes time to execute that SQL statement. This will specifically be required for SQLite primary key modifications in modern versions of SQLite and Django. Now, rather than just returning hard-coded SQL statements, an `SQLResult`'s list of statements can now contain a callback function that takes a cursor and returns new statements. This will be called when it'd otherwise be time to execute the SQL statement for that entry in the list. To make this happen, some part of the execution/logging code had to change. Previously, we'd normalize the list of SQL statements for output, and then normalize a different way for execution (the main differences between that we'd apply parameters to format strings for logging, but pass them separately to the database backend for execution). Since we need the callbacks to execute only once, these operations had to merge. To do this, there's now a private function dedicated to taking a list of SQL statements/callbacks and yielding results one-by-one. There's then a function, `run_sql()`, which provides the main logic for both execution and capturing for logging, allowing both to happen in one pass. The original `execute_sql()` and `write_sql()` methods wrap this, behaving as before but, in the case of `execute_sql()`, allowing the caller to capture as part of the standard execution process.
    c33e6b7af0baac85df43ca69a60b23ebf54ef864
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to master (2998abc)