Allow for parallel full-repo builds on the same repository on Git.

Review Request #12136 — Created March 11, 2022 and submitted

Information

ReviewBot
release-3.0.x

Reviewers

The Git checkout process attempted to create a branch for the purpose
of cloning a working area. Git doesn't provide any official, reliable
mechanism for cloning form a specific commit, so we first create the
branch on the desired commit and then clone that.

This ultimately doesn't work well, since if there are two tools that
both need a workarea from the same commit, they can end up trying to
create, clone from, and destroy the same exact branch, leading to race
conditions.

We now include a UUID along with the branch name, which should help
mitigate this issue. We also attempt to destroy the branch after we
clone even if that clone operation fails.

We also no longer prune branches when fetching, which led to another
race condition where in-progress branches could be removed prematurely.

Tested with a bunch of parallel tool runs across multiple workers with
a shared set of repository directories. No longer encountered errors
creating branches, cloning, or cleaning up branches.

Summary ID
Allow for parallel full-repo builds on the same repository on Git.
The Git checkout process attempted to create a branch for the purpose of cloning a working area. Git doesn't provide any official, reliable mechanism for cloning form a specific commit, so we first create the branch on the desired commit and then clone that. This ultimately doesn't work well, since if there are two tools that both need a workarea from the same commit, they can end up trying to create, clone from, and destroy the same exact branch, leading to race conditions. We now include a UUID along with the branch name, which should help mitigate this issue. We also attempt to destroy the branch after we clone even if that clone operation fails. We also no longer prune branches when fetching, which led to another race condition where in-progress branches could be removed prematurely.
ba0c628e11e42fb6b0df5865fce0a6c4b3895aaf
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-3.0.x (f5a2c56)