Redesign rbdemo fixture loading for safety and better maintenance.

Review Request #12403 — Created June 23, 2022 and submitted — Latest diff uploaded

Information

rb-extension-pack
master

Reviewers

rbdemo has always been a bit hard to maintain. We relied on Django's
fixture support to dump out state and load it in, and this doesn't
really work well when moving between versions of Django or Review Board.
It's also just really hard to maintain the data we want to put up.

This is a complete redesign of how we deal with state for rbdemo.

First, safety. We no longer use Django's management commands to reset
database state. These run in, and commit, their own transactions,
meaning that if we fail to load state after we reset, we just end up
with an empty database.

We now handle the flushing/resetting process ourselves, but more
selectively. Tables essential to running the server (site, siteconfig,
evolutions, migrations, etc.) are kept intact. This also makes it much
easier to set up a mock demo server for development.

Everything is now run in a single transaction. If anything goes wrong,
we shouldn't lose anything already in the database. This makes periodic
resetting safe.

The idea of how we load data has also been entirely redone.

We no longer use Django's fixtures support. Instead, we load a YAML file
(or files) that contains basic structured data needed to populate a demo
server. This is easier to maintain, as we can just edit some content in
the files and re-import into a demo server.

Since we're in full control of the loading process now, and are less
tied to exact database state, we should have a lot fewer problems
updating this as we upgrade Review Board.

Tested reloading demo data with errors. The server no longer had to be
reset.

Built a YAML document containing all the content from demo.reviewboard.org
and have successfully been able to import it, edit it, and re-import it
numerous times. The content on the server fully matched expectations.

Changes between revision 1 and 2

orig
1
2
3

Commits

Summary ID Author
Redesign rbdemo fixture loading for safety and better maintenance.
rbdemo has always been a bit hard to maintain. We relied on Django's fixture support to dump out state and load it in, and this doesn't really work well when moving between versions of Django or Review Board. It's also just really hard to maintain the data we want to put up. This is a complete redesign of how we deal with state for rbdemo. First, safety. We no longer use Django's management commands to reset database state. These run in, and commit, their own transactions, meaning that if we fail to load state after we reset, we just end up with an empty database. We now handle the flushing/resetting process ourselves, but more selectively. Tables essential to running the server (site, siteconfig, evolutions, migrations, etc.) are kept intact. This also makes it much easier to set up a mock demo server for development. Everything is now run in a single transaction. If anything goes wrong, we shouldn't lose anything already in the database. This makes periodic resetting safe. The idea of how we load data has also been entirely redone. We no longer use Django's fixtures support. Instead, we load a YAML file (or files) that contains basic structured data needed to populate a demo server. This is easier to maintain, as we can just edit some content in the files and re-import into a demo server. Since we're in full control of the loading process now, and are less tied to exact database state, we should have a lot fewer problems updating this as we upgrade Review Board.
997fa3bfc1abbd50cef87dd06d2b8a6e0a53ad5d Christian Hammond
Redesign rbdemo fixture loading for safety and better maintenance.
rbdemo has always been a bit hard to maintain. We relied on Django's fixture support to dump out state and load it in, and this doesn't really work well when moving between versions of Django or Review Board. It's also just really hard to maintain the data we want to put up. This is a complete redesign of how we deal with state for rbdemo. First, safety. We no longer use Django's management commands to reset database state. These run in, and commit, their own transactions, meaning that if we fail to load state after we reset, we just end up with an empty database. We now handle the flushing/resetting process ourselves, but more selectively. Tables essential to running the server (site, siteconfig, evolutions, migrations, etc.) are kept intact. This also makes it much easier to set up a mock demo server for development. Everything is now run in a single transaction. If anything goes wrong, we shouldn't lose anything already in the database. This makes periodic resetting safe. The idea of how we load data has also been entirely redone. We no longer use Django's fixtures support. Instead, we load a YAML file (or files) that contains basic structured data needed to populate a demo server. This is easier to maintain, as we can just edit some content in the files and re-import into a demo server. Since we're in full control of the loading process now, and are less tied to exact database state, we should have a lot fewer problems updating this as we upgrade Review Board. The old `dump-demo-data` command is now gone, as the YAML data is now a source document and not just a database backup.
ec432138d9334e71b916dad51b06031fb71826fb Christian Hammond
rbdemo/rbdemo/management/commands/dump-demo-data.py
rbdemo/rbdemo/management/commands/reset-demo.py
Loading...