• 
      

    Fix processing of non-UTF-8-encoded files and diffs

    Review Request #872 — Created May 20, 2009 and submitted — Latest diff uploaded

    Information

    Review Board SVN (deprecated)

    Reviewers

    When files in a repository is encoded with a non-ASCII, non-UTF-8 encoding, a special configuration option, repository encoding is required. However even if this option is provided files are still processed incorrectly by diffviewer.
    
    convert_to_utf8() correctly returns unicode strings for byte strings which can be decoded as UTF-8 (i.e. ASCII and actual UTF-8) and further processing (e.g. by pygments) assumes unicode strings as parameters. However for non-UTF-8 strings the function returned byte strings which effectively break pygments.
    
    The patch
    1. renames convert_to_utf8() to convert_to_unicode() to reflect its real purpose :)
    2. return unicode instead of str for strings in a user-specified encoding