Improve the performance of HostingServiceClient._build_form_data

Review Request #8910 - Created April 21, 2017 and submitted

Barret Rennie
Review Board
release-2.5.x
8909
8911
reviewboard

The method in question was doing multiple string concatenations with
+=, which is very inefficient for large number of operations becuase
each operation results in a memory allocation. We now build a single
string with string interpolation for each field and file passed in and
join all the blobs together, which will require only a single memory
allocation for the final string, instead of O(|fields| + |files|)
allocations.

Ran unit tests.

  • 0
  • 0
  • 8
  • 0
  • 8
Description From Last Updated
Christian Hammond
  1. 
      
  2. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    While here, we should also make this a byte string.

    Here and below.

  3. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    Let's just make this:

    if files:
    

    Covers None and empty dictionaries.

  4. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     
     
     

    Let's also use six.iteritems here, pull out the value only once and during iteration, to speed things up a little more.

  5. reviewboard/hostingsvcs/service.py (Diff revision 1)
     
     

    Trailing comma.

  6. 
      
Barret Rennie
Barret Rennie
David Trowbridge
  1. 
      
  2. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     

    We should check if these are unicode and if so, encode to utf-8 before formatting.

  3. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     
     

    Same here.

  4. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     
     

    Can we switch these two lines to match the order in the return statement?

  5. reviewboard/hostingsvcs/service.py (Diff revision 2)
     
     

    Should be b''.join(...)

  6. 
      
Barret Rennie
David Trowbridge
  1. Ship It!
  2. 
      
Barret Rennie
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-2.5.x (1579fb5)
Loading...