Fix critical crash when running on non utf-8 environment.

Review Request #7395 — Created June 8, 2015 and submitted

Information

RBTools

Reviewers

When running on non UTF-8 environment with mercurial as repository RBTools crashes for every command with message:
'CRITICAL: 'utf8' codec can't decode byte 0xb3 in position 22: invalid start byte.'. This is caused by assumption in process.py execute method.

Now we are using sys.getfilesystemencoding() instead of assumpted 'utf-8'.

Running on non UTF-8 environment (Win7, x64 PL) does not cause error anymore.

Description From Last Updated

Blank line between these.

chipx86chipx86

The filesystem encoding doesn't seem like the "correct" thing here, since this is the output of the program. Does sys.getdefaultencoding() …

daviddavid
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        rbtools/utils/process.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        rbtools/utils/process.py
    
    
  2. 
      
chipx86
  1. Mind reworking the description into more of the format described here? https://www.reviewboard.org/docs/codebase/dev/writing-good-descriptions/
    That will help with getting a thorough understanding of the problem, its cause, its fix, and why the fix works. It will also help when bisecting commits later, and with preparation of release notes.

  2. rbtools/utils/process.py (Diff revision 1)
     
     
     
    Show all issues

    Blank line between these.

  3. 
      
BG
BG
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        rbtools/utils/process.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        rbtools/utils/process.py
    
    
  2. 
      
david
  1. r2 of your diff doesn't seem like it is correct (it seems to only be the top commit instead of the full diff from origin/master)

  2. rbtools/utils/process.py (Diff revision 2)
     
     
    Show all issues

    The filesystem encoding doesn't seem like the "correct" thing here, since this is the output of the program. Does sys.getdefaultencoding() return the right thing?

    1. I think it does. Its better than assuming 'utf-8' (which is incorrect for windows family os [mbsc]).
      
      Most correctly wrriten apps should use filesystem encoding to output, due to pipelining, streaming to file.
      
      Win7: sys.getfilesystemencoding() returns mbsc
      Debian: sys.getfilesystemencoding() returns UTF-8
      
      sys.getdefaultencoding() always returns ascii, except when during app init it was changed by sys.setdefaultencoding() (which is 'discouraged'). See: http://stackoverflow.com/questions/3828723/why-we-need-sys-setdefaultencodingutf-8-in-a-py-script
    2. Sounds good, thanks for the explanation.

  3. 
      
BG
reviewbot
  1. Tool: Pyflakes
    Processed Files:
        rbtools/utils/process.py
    
    
    
    Tool: PEP8 Style Checker
    Processed Files:
        rbtools/utils/process.py
    
    
  2. 
      
david
  1. Ship It!
  2. 
      
BG
Review request changed
Status:
Completed
Change Summary:
Pushed to release-0.7.x (df994eb)