• 
      

    Optimize diff processing for Subversion.

    Review Request #12640 — Created Sept. 25, 2022 and submitted — Latest diff uploaded

    Information

    RBTools
    release-4.x

    Reviewers

    When generating Subversion diffs, RBTools does a lot of diff processing
    to:

    1. Ensure that added/deleted empty files are present
    2. Ensure renamed files have the right information
    3. Convert relative paths to absolute paths
    4. Filter for any excluded files

    Each step of this would iterate over the previous diff and then generate
    a new one. If the diff had, say, 10,000 lines, we'd parse and then
    re-build those 10,000 lines in almost every one of those stages, before
    finally returning the result. This was slow.

    This change converts each processing stage to a generator, allowing one
    pass through the diff. Each stage will iterate through and yield lines
    for the next stage, the result of which is iteratively joined into a
    final byte string.

    The stages themselves have been optimized a bit. We used to perform
    repeated lookups on SVNClient for the same attributes (compiled
    regexes and constants) on each loop. Those are now pulled out of the
    loops.

    A potential bug was also found and fixed that could have led to an
    infinite loop during diff generation when processing empty files. If a
    very specific (unlikely) set of conditions were met (deleted empty file,
    revision values of 0 for the diff operation, and no Revision found for
    the file in svn info), we'd repeate the loop, but with no advancement.
    That'd cause the same condition to be hit over and over. We now advance
    the iterator.

    Unit tests pass.

    Commits

    Files