• 
      

    Fix applying Pygments syntax highlighting to diffs on Python 3.

    Review Request #10708 — Created Sept. 10, 2019 and submitted

    Information

    Review Board
    release-4.0.x
    80282b4...

    Reviewers

    A string type compatibility issue on Python 3 resulted in a failure to
    apply syntax highlighting using Pygments. After generating the
    syntax-highlighted HTML (which is returned as a Unicode string), we
    attempted to split the lines using split_line_endings() (which
    required a byte string), and the type discrepency resulted in an
    exception that was caught and ignored, causing the diff viewer to fall
    back on a non-highlighted view.

    To address this, split_line_endings() has been made compatible with
    both byte and Unicode strings, and we no longer blindly ignore all
    exceptions. Now, the only exception that is ignored is Pygments'
    ClassNotFound, which is used when it can't determine a lexer for the
    file type.

    As part of the fix for exception handling, the code has been reworked to
    both handle exceptions and to look up files in the highlighting
    blacklist entirely within _apply_pygments(), to simplify the
    requirements for the caller and to ensure that if the left-hand side of
    the diff fails to syntax-highlight, it won't prevent the right-hand side
    from trying.

    Unit tests have been added to ensure that both split_line_endings()
    and _apply_pygments() will do their jobs correctly, and that the final
    chunk results from RawDiffChunkGenerator contain the resulting lines
    we expect.

    All unit tests passed on Python 2.7, 3.5, 3.6, and 3.7.

    Verified that syntax highlighting was functioning properly.

    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-4.0.x (4a20674)