Make the upload location of avatars configurable

Review Request #8841 — Created March 25, 2017 and submitted

Information

Djblets
release-0.10.x
45052d2...

Reviewers

The upload location of avatars via the FileUploadService is now
configurable through the UPLOADED_AVATARS_PATH Django setting and the
get_unique_filename method. If the UPLOADED_AVATARS_PATH setting is
not set, a default of uploaded/avatars will be used.

To enable this customizability, AvatarServiceConfigForms are now
passed the instance of the service that instantiated them as a keyword
argument. This allows the forms to look up static settings on the
services.

Ran unit tests.

Description From Last Updated

Why not just put service=None into the method signature?

daviddavid

Can you not fit this all on one line?

daviddavid

, optional

chipx86chipx86

I'd like to propose some changes to this logic that will better handle larger files and let Django backends retain …

chipx86chipx86

We should make sure to add a / if file_path_prefix doesn't end with it. It's safe to use this as …

chipx86chipx86

Contents are indented 1 space too far.

chipx86chipx86

We don't use "Returns" for properties.

chipx86chipx86

We should use os.path.join for the uploaded/avatars/ (which also shouldn't have a leading /).

chipx86chipx86

Mind using keyword arguments for this one too?

chipx86chipx86

Typo.

chipx86chipx86

Should just import os. os.path is implied when doing that.

chipx86chipx86

We access the username 3 times. Let's pull it out into a variable to avoid accessor lookups.

chipx86chipx86

No need for the 0.

chipx86chipx86

Minor thing, but for consistency... Most classes use "Unit tests for ..."

chipx86chipx86

Looks unintentional. Blank line should remain.

chipx86chipx86
david
  1. 
      
  2. djblets/avatars/forms.py (Diff revision 1)
     
     
     
    Show all issues

    Why not just put service=None into the method signature?

  3. djblets/avatars/services/file_upload.py (Diff revision 1)
     
     
     
     
    Show all issues

    Can you not fit this all on one line?

  4. 
      
brennie
chipx86
  1. 
      
  2. djblets/avatars/forms.py (Diff revision 2)
     
     
    Show all issues

    , optional

  3. djblets/avatars/services/file_upload.py (Diff revision 2)
     
     
     
     
     
     
     
     
     
     
     
    Show all issues

    I'd like to propose some changes to this logic that will better handle larger files and let Django backends retain control over the saving process a bit better.

    First, it's generally better not to read in all the file data at once. Imagine if someone tried to upload a 100MB file as an avatar. We'd try to read it in all at once and swamp the server, crashing things and effectively taking Review Board down. Instead, we can use chunking to build the hash, like so:

    uploaded_file = self.cleaned_data['avatar_upload']
    file_hash = md5()
    
    for chunk in uploaded_file.chunks():
        file_hash.update(chunk)
    

    For path building, there's a couple things we need to do. Paths are expected to go in as OS-specific paths, and are then normalized in the backend, so we can use os.path.join. We'll want to do this for the default value fo file_path_prefix (I'll cover this in another comment) and we want to build it here, so that we can be sure things work if file_path_prefix doesn't leave off the trailing /.

    For saving, we should let Storage.save do its job. This is a bit better than using open and write, and gives the backends a lot more control of the process (and is, I think, actually necessary for some backends). By default, save will ensure we don't end up with a duplicate filename under any circumstances. It will also ensure paths exist, and it will handle chunking (like above). So the rest of this logic in this comment should be:

    file_path = storage.save(
        os.path.join(
            self.service.file_path_prefix,
            self.service.get_unique_filename(
                storage.get_valid_filename(filename))),
        uploaded_file)
    

    We're using get_valid_name on the user-provided filename, instead of the resulting path, helping to catch problems, and doing the hash thing there. We're then creating an OS-level path, which storages expect. The result from the method will be the normalized file path that we can return (which always uses "/" for the delimiter, but also takes care of any alterations that need to be made to make the path more correct).

    Give that a try. It should result in better future-compatible avatar saving.

  4. djblets/avatars/services/file_upload.py (Diff revision 2)
     
     
    Show all issues

    We should make sure to add a / if file_path_prefix doesn't end with it.

    It's safe to use this as the delimiter for paths in storage. Django will normalize to this on Windows, anyway.

  5. djblets/avatars/services/file_upload.py (Diff revision 2)
     
     
     
     
     
     
     
     
     
     
    Show all issues

    Contents are indented 1 space too far.

  6. djblets/avatars/services/file_upload.py (Diff revision 2)
     
     
     
     
     
    Show all issues

    We don't use "Returns" for properties.

  7. djblets/avatars/services/file_upload.py (Diff revision 2)
     
     
    Show all issues

    We should use os.path.join for the uploaded/avatars/ (which also shouldn't have a leading /).

  8. 
      
brennie
brennie
chipx86
  1. 
      
  2. djblets/avatars/services/base.py (Diff revision 4)
     
     
    Show all issues

    Mind using keyword arguments for this one too?

  3. djblets/avatars/services/base.py (Diff revision 4)
     
     
    Show all issues

    Typo.

  4. djblets/avatars/services/file_upload.py (Diff revision 4)
     
     
    Show all issues

    Should just import os. os.path is implied when doing that.

  5. djblets/avatars/services/file_upload.py (Diff revision 4)
     
     
     
    Show all issues

    We access the username 3 times. Let's pull it out into a variable to avoid accessor lookups.

  6. djblets/avatars/services/file_upload.py (Diff revision 4)
     
     
    Show all issues

    No need for the 0.

  7. 
      
brennie
brennie
chipx86
  1. Ship It!
  2. 
      
brennie
brennie
brennie
brennie
chipx86
  1. 
      
  2. djblets/avatars/tests.py (Diff revision 10)
     
     
    Show all issues

    Minor thing, but for consistency... Most classes use "Unit tests for ..."

  3. djblets/configforms/forms.py (Diff revision 10)
     
     
     
     
     
    Show all issues

    Looks unintentional. Blank line should remain.

  4. 
      
brennie
chipx86
  1. Ship It!
  2. 
      
brennie
Review request changed
Status:
Completed
Change Summary:
Pushed to release-0.10.x (bd627ac)