Improve the format and capabilities for mapped SQL test statements.

Review Request #11077 — Created July 13, 2020 and submitted

Information

Django Evolution
master

Reviewers

Django Evolution makes use of a large number of SQL statements that
generated SQL is compared against, one collection per supported
database. These were previously strings that were split up at comparison
time. In the test data these were often represented as lines that were
joined together into single strings.

Instead of doing the monotonous work of joining lists of strings into
single strings and then splitting again, we're now using lists of
strings everywhere. This is easier to maintain, eliminates subtle
problems encountered in the past (where a trailing comma on a string
from a copy/paste would result in a tuple, causing breakages), and
allows us to perform pattern matching.

Now, these statements can include a re.compile(...) in place of a
string in order to pattern match against generated SQL. While we always
prefer to perform exact matches, this is important for some upcoming
code that generates SQL containing a value made available by the
database itself that we cannot fake or safely influence.

If a regex is found, it triggers slightly-more-involved SQL comparisons,
but the output is generally the same.

To aid with all this, the assertion code that compares against these SQL
statements has been pulled out into an assertSQLMappingEqual()
statement, which some tests have been updated to use instead of trying
to perform their own normalization and assertions. These assertions are
now standardized and easy to bring into new tests in the future.

Unit tests pass for all database types on Python 2.7 and 3.8 on
Django 1.11 and 2.0.

Other versions were not tested as this code isn't very dependent on the
versions of the dependencies used.

Summary ID
Improve the format and capabilities for mapped SQL test statements.
Django Evolution makes use of a large number of SQL statements that generated SQL is compared against, one collection per supported database. These were previously strings that were split up at comparison time. In the test data these were often represented as lines that were joined together into single strings. Instead of doing the monotonous work of joining lists of strings into single strings and then splitting again, we're now using lists of strings everywhere. This is easier to maintain, eliminates subtle problems encountered in the past (where a trailing comma on a string from a copy/paste would result in a tuple, causing breakages), and allows us to perform pattern matching. Now, these statements can include a `re.compile(...)` in place of a string in order to pattern match against generated SQL. While we always prefer to perform exact matches, this is important for some upcoming code that generates SQL containing a value made available by the database itself that we cannot fake or safely influence. If a regex is found, it triggers slightly-more-involved SQL comparisons, but the output is generally the same. To aid with all this, the assertion code that compares against these SQL statements has been pulled out into an `assertSQLMappingEqual()` statement, which some tests have been updated to use instead of trying to perform their own normalization and assertions. These assertions are now standardized and easy to bring into new tests in the future.
0df2986357bb639f8822d5a728265557ac135cde
Description From Last Updated

W503 line break before binary operator

reviewbotreviewbot

E127 continuation line over-indented for visual indent

reviewbotreviewbot
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

chipx86
david
  1. Nice cleanup!

  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to master (f28e57a)