Fix applying Pygments syntax highlighting to diffs on Python 3.
Review Request #10708 — Created Sept. 10, 2019 and submitted
A string type compatibility issue on Python 3 resulted in a failure to
apply syntax highlighting using Pygments. After generating the
syntax-highlighted HTML (which is returned as a Unicode string), we
attempted to split the lines using
required a byte string), and the type discrepency resulted in an
exception that was caught and ignored, causing the diff viewer to fall
back on a non-highlighted view.
To address this,
split_line_endings()has been made compatible with
both byte and Unicode strings, and we no longer blindly ignore all
exceptions. Now, the only exception that is ignored is Pygments'
ClassNotFound, which is used when it can't determine a lexer for the
As part of the fix for exception handling, the code has been reworked to
both handle exceptions and to look up files in the highlighting
blacklist entirely within
_apply_pygments(), to simplify the
requirements for the caller and to ensure that if the left-hand side of
the diff fails to syntax-highlight, it won't prevent the right-hand side
Unit tests have been added to ensure that both
_apply_pygments()will do their jobs correctly, and that the final
chunk results from
RawDiffChunkGeneratorcontain the resulting lines
All unit tests passed on Python 2.7, 3.5, 3.6, and 3.7.
Verified that syntax highlighting was functioning properly.