Summary

Fix UnicodeDecodeError when commit message contains non-ascii character

Review Request #10121 — Created Aug. 15, 2018 and discarded Oct. 29, 2018, 2:31 p.m.

Information

Owner

fangwentong

Repository

RBTools

Branch

master

Bugs

Depends On

Reviewers

Groups

rbtools

People

Description

subprocess output data is str type, convert it to unicode when results_unicode is true

Testing Done


Posted some changes.
Ran unit tests.

Issues

Description	From	Last Updated
When results_unicode is True, subprocess is called with universal_newlines=True. In this case, it should return results as unicode instead of …	david	Aug. 30, 2018, 7:41 p.m.
E501 line too long (85 > 79 characters)	reviewbot	Aug. 15, 2018, 11:57 p.m.

flake8 failed.

JSHint passed.

flake8

rbtools/utils/commands.py (Diff revision 1)
The issue has been resolved. Show all issues
```
E501 line too long (85 > 79 characters)
```

Commit:

c62c7a2fd48eb9f23794aef00a59469b2463fab2

f6ddaad3796e356c3dc7b81d2539662909b03bf8

Diff:

Revision 2 (+3 -2)

Show changes

rbtools/utils/commands.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Commit:

f6ddaad3796e356c3dc7b81d2539662909b03bf8

b99e3d8179a25e26dc69ffe64dcb59590b495200

Diff:

Revision 3 (+3)

Show changes

rbtools/utils/commands.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Commit:

b99e3d8179a25e26dc69ffe64dcb59590b495200

e15fd2d87df5cdcdca29280a77c49344d7b3f365

Diff:

Revision 4 (+3)

Show changes

rbtools/utils/commands.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Testing Done:: +
testing done with python2.7.15 and python 3.7.0

Testing Done:: ~
testing done with python2.7.15 and python 3.7.0
~
testing done under python2.7.15 and python 3.7.0

Summary:

Fix UnicodeDecodeError when stamp commit message with non-ascii character

Fix UnicodeDecodeError when commit message contains non-ascii character

Description:

~		commit_message returned by GitClient.get_raw_commit_message() is str type, stamp_string is unicode type.
	~	subprocess output data is str type, convert it to unicode when results_unicode is true
-
-		since default encoding is 'ascii', if the commit_message contains non-ascii character,
-		statement `if stamp_string in commit_message:` will throw UnicodeDecodeError

Testing Done:

~		testing done under python2.7.15 and python 3.7.0
	~	Posted some changes.
	+	Ran unit tests.

Commit:

e15fd2d87df5cdcdca29280a77c49344d7b3f365

208eae4651a8ebca4c452fb3fcff2a1c9b978521

Diff:

Revision 5 (+14 -1)

Show changes

	rbtools/utils/console.py
	rbtools/utils/process.py
	rbtools/utils/tests.py

Checks run (1 succeeded, 1 failed with error)

flake8 internal error.

JSHint passed.

Commit:

208eae4651a8ebca4c452fb3fcff2a1c9b978521

ada54f6934d45141d23ddd35c6e00330f619d5a1

Diff:

Revision 6 (+14 -1)

Show changes

	rbtools/utils/console.py
	rbtools/utils/process.py
	rbtools/utils/tests.py

Checks run (1 succeeded, 1 failed with error)

flake8 internal error.

JSHint passed.

Commit:

ada54f6934d45141d23ddd35c6e00330f619d5a1

652624311195820212f58fa5a638d6ddabbc8e73

Diff:

Revision 7 (+4 -1)

Show changes

	rbtools/utils/console.py
	rbtools/utils/process.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

The issue has been resolved. Show all issues

When results_unicode is True, subprocess is called with universal_newlines=True. In this case, it should return results as unicode instead of bytes. Is there a particular case you're seeing where this isn't working?

fangwentong Aug. 30, 2018, 7:41 p.m.

Yes, I tested this on MacOS and ubuntu with python2.7, subprocess.communicate() returns str instead of unicode.

when commit message contains non-ascii character, rbt post --stamp will raise UnicodeDecodeError and fail add stamp to commit message.

see the test case in diff 6: https://reviews.reviewboard.org/r/10121/diff/6/. this testcase cause flake8 internal error, so I remove the testcase in diff 7

fangwentong Aug. 30, 2018, 7:50 p.m.

When commit message contains non-ascii character, rbt post -r will fail and prompt UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128).

Modification in rbtools/utils/console.py fixed this problem.

Status:: Discarded

~		testing done with python2.7.15 and python 3.7.0
	~	testing done under python2.7.15 and python 3.7.0

rbtools/utils/commands.py (Diff revision 1)