• 
      

    Add a streaming parser for DiffX files.

    Review Request #11712 — Created July 7, 2021 and submitted — Latest diff uploaded

    Information

    DiffX
    master

    Reviewers

    This introduces diffx.reader.DiffXReader, a streaming parser for the
    DiffX file format. This is a low-level interface for DiffX files, which
    is able to read from a file stream (such as a local file, HTTP response,
    or memory-backed stream) and parse and return each section in the DiffX
    files according to the specification.

    The parser acts as a generator, and will provide dictionaries containing
    information on each section of the DiffX file. This includes options,
    metadata or text/diff content, the section ID, section type, and section
    hierarchy level. Consumers can build upon this to work with the data as
    it comes in, or to easily convert it into another representation.

    As a streaming parser, this does not keep much state around between
    sections. This makes it ideal for working with very large DiffX files,
    or reading from a stream that may incrementally provide new content.
    However, it also means that a parsing error may not manifest until later
    in the stream, after the consumer has already handled some content. An
    object-based implementation is in the works that will address this.

    As this is a reference implementation, the parser is very strict about
    conforming to the specification, when it comes to header structure,
    characters in header options, lengths, and valid option values. It does
    take into consideration different newline formats and ignores extra
    newlines before/after sections (including at the beginning or end of the
    file).

    At this stage, DiffXReader should be usable for any production use of
    the DiffX format. An accompanying DiffXWriter, and implementations
    representing DiffX through an object model, are in the works.

    Unit tests pass on Python 2 and 3.

    Put the reader through its paces by testing a bunch of valid and invalid
    DiffX files generated by hand and by the upcoming DiffXWriter.

    Commits

    Files