Improve the performance of HostingServiceClient._build_form_data

Review Request #8910 — Created April 21, 2017 and submitted

brennie
Review Board
release-2.5.x
8909
8911
reviewboard

The method in question was doing multiple string concatenations with
+=, which is very inefficient for large number of operations becuase
each operation results in a memory allocation. We now build a single
string with string interpolation for each field and file passed in and
join all the blobs together, which will require only a single memory
allocation for the final string, instead of O(|fields| + |files|)
allocations.

Ran unit tests.

Description From Last Updated

While here, we should also make this a byte string. Here and below.

chipx86chipx86

Let's just make this: if files: Covers None and empty dictionaries.

chipx86chipx86

Let's also use six.iteritems here, pull out the value only once and during iteration, to speed things up a little ...

chipx86chipx86

Trailing comma.

chipx86chipx86

We should check if these are unicode and if so, encode to utf-8 before formatting.

daviddavid

Same here.

daviddavid

Can we switch these two lines to match the order in the return statement?

daviddavid

Should be b''.join(...)

daviddavid
chipx86
  1. 
      
  2. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    While here, we should also make this a byte string.

    Here and below.

  3. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    Let's just make this:

    if files:
    

    Covers None and empty dictionaries.

  4. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     
     
     

    Let's also use six.iteritems here, pull out the value only once and during iteration, to speed things up a little more.

  5. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    Trailing comma.

  6. 
      
brennie
brennie
david
  1. 
      
  2. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     

    We should check if these are unicode and if so, encode to utf-8 before formatting.

  3. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     
     

    Same here.

  4. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     

    Can we switch these two lines to match the order in the return statement?

  5. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     

    Should be b''.join(...)

  6. 
      
brennie
david
  1. Ship It!
  2. 
      
brennie
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-2.5.x (1579fb5)
Loading...