After you've fixed errors, please marked them as such. Our infrastructure doesn't allow patches to be landed with open issues.

It should be noted that mercurial.py doesnt use unicode_literals yet.

Oh shucks. We should do that.

From my understanding in order to convert the unicode string into a byte array you need to pass in an encoding: unicode_string.decode(encoding). I'm afraid I'm not a Python expert, not sure how to do this properly without breaking non-utf8 files.

At a basic level, we can just prefix each of these strings with a b (i.e. b'diff -r %s -r %s %s\n'). We also need to make sure that the things that get formatted into the string (base, tip, filename, etc.) are all bytestrings too.

It seems that doesn't work. In this case the problem is the diff variable is a unicode string, where as 'diff -r ...' is not. Adding the b prefix would cause the non-unicode string to be converted to a byte array, and then the byte array is being added to a unicode string which ends up causing the same UnicodeDecodeError. diff needs to be converted to a byte array so that it can be manipulated, and then back to a unicode string before the end of _handle_empty_files.
I tried the following as well as b'diff -r ...':

diff += (u'diff -r %s -r %s %s\n'

         u'--- %s\t%s\n'

         u'+++ b/%s\t%s\n'

         % (base, tip, filename,

            self.PRE_CREATION, self.PRE_CREATION_DATE,

            filename, tip_date))
In both cases I still get a UnicodeDecodeError.

Ali,
After looking at the code, I think there's probably a fair amount of refactoring that will have to happen here, so I'm going to take over the fix for this. Thanks for raising the issue!

Thanks David!