Fix applying Pygments syntax highlighting to diffs on Python 3.
Review Request #10708 — Created Sept. 10, 2019 and submitted — Latest diff uploaded
A string type compatibility issue on Python 3 resulted in a failure to
apply syntax highlighting using Pygments. After generating the
syntax-highlighted HTML (which is returned as a Unicode string), we
attempted to split the lines usingsplit_line_endings()
(which
required a byte string), and the type discrepency resulted in an
exception that was caught and ignored, causing the diff viewer to fall
back on a non-highlighted view.To address this,
split_line_endings()
has been made compatible with
both byte and Unicode strings, and we no longer blindly ignore all
exceptions. Now, the only exception that is ignored is Pygments'
ClassNotFound
, which is used when it can't determine a lexer for the
file type.As part of the fix for exception handling, the code has been reworked to
both handle exceptions and to look up files in the highlighting
blacklist entirely within_apply_pygments()
, to simplify the
requirements for the caller and to ensure that if the left-hand side of
the diff fails to syntax-highlight, it won't prevent the right-hand side
from trying.Unit tests have been added to ensure that both
split_line_endings()
and_apply_pygments()
will do their jobs correctly, and that the final
chunk results fromRawDiffChunkGenerator
contain the resulting lines
we expect.
All unit tests passed on Python 2.7, 3.5, 3.6, and 3.7.
Verified that syntax highlighting was functioning properly.