Switch our media setup to use Django's staticfiles support.

Review Request #2914 — Created Feb. 23, 2012 and submitted

Information

Review Board
master

Reviewers

Switch our media setup to use Django's staticfiles support.

Django introduced built-in support for static media files in Django 1.3,
and have been improving upon this in 1.4. This staticfiles support makes
it possible for a site to specify where to look for media files (in app
directories, specific directories, or through custom Finder classes)
and, with a 'collectstatic' command, deploy them somewhere (another
directory or some other location through a Storage module, such as
Amazon S3). It also provides built-in support for post-processing
modules.

Django Pipeline now requires this support to even operate, and Django
1.4 is very strict about STATIC_ROOT not being empty and not being equal
to MEDIA_ROOT. So it's necessary that we move along with the world,
painful as it may be.

In this new world, there's a clear distinction between STATIC_* and
MEDIA_*. STATIC_* is used for media bundled with the application (our
own CSS, JavaScript, etc.), while MEDIA_* is for user-uploaded media
(screenshots, file attachments, etc.). This means that we can actually
in theory dump the files into different places (though maybe not yet in
practice).

One advantage this all does give us is that our initial setup is
simpler. Instead of prepare-dev having to get the djblets symlinks in
the right place, we can just rely on the staticfiles stuff to pull it in
from wherever it's installed. It also means that we can simplify our
unit tests' media management.

This will require a lot of testing and will likely break in some
circumstances.

One thing that users will have to deal with when upgrading is to
update their Apache/server config file to set up an alias for
/static. Otherwise, they won't see their media files. It's an annoyance,
and one that we can't fix for them. I have plans to switch to a new
set of config files, though, which will allow us to make updates
like this without breaking users later, and so this is a good time
to force them into making changes.
Tested various Review Board pages, including the login page, dashboard,
New Review Request page, review requests, screenshots, diff viewer, and
admin UI.

Tested media with DEBUG and non-DEBUG builds, and with DEBUG but turning
on media pipelining.

Tested creating a new site and verifying that the media dirs were still
symlinking as expected into the locations specified by the user.

Tested upgrading a site and verified that the media worked as expected (after
manually updating the Apache config file to point to the static directory),
and that the old static media links were gone.

Tested that an old configuration with in-database site_media_url and
site_media_root paths were updated to create accompanying static versions.

All Unit tests passed.
Description From Last Updated

Indentation here is funky

daviddavid
david
  1. I didn't see too much in rbsite.py that looked scary, but do we need to warn upgraders about anything?
    1. Yeah, basically, sites will be broken until their Apache configs are updated. My plan of action is this:
      
      1) Take the opportunity to restructure our web server config files. Create two files: The ones we have today, and "common" files per server which we can update later. The "common" files will be included in the main server files, and will have stuff like the Aliases and <Directory> settings and Expiration and stuff, allowing us to maintain this stuff down the road. It'll be expected that users will use our includes from here on out.
      
      2) Get rb-site to offer to regenerate the existing config files, overwriting their settings. If they're just symlinking into their Apache config directory, they should just get the new updates. Otherwise, they'll at least have something they can look at to base their new changes off of.
      
      3) Maybe have some sort of environment variable we set in the config files indicating, I dunno, some config version or something. We can use this in our Manual Updates checks. If it's set and is what we expect, we're good. Otherwise, we present a Manual Updates page and tell them what went wrong and how to fix it (namely, tell them to include our new common files).
      
      How does that sound? I'm not in love with breaking installs with this upgrade, but we're a bit stuck there, and at least we can work toward a better setup for future breakages.
    2. That sounds good.
  2. reviewboard/urls.py (Diff revision 1)
     
     
     
     
     
    Show all issues
    Indentation here is funky
  3. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to master (882c752)