Skip SSH tests when a system-wide executable is missing.

Review Request #12147 — Created March 14, 2022 and submitted

Review Board

A long-standing, annoying issue that has plagued many of us developing
on Review Board have been Bazaar SSH unit tests failing with Repository
Not Found errors.

This is actually due to Bazaar not being installed locally in a
directory part of the system path established when running a command via
SSH, without the shell's configuration setting up a normal path. For
instance, ssh localhost bzr won't result in a ~/.zshrc being
executed (just a ~/.zshenv). It certainly won't be found in a
virtualenv. This leads to the error.

This is actually not limited to Bazaar. CVS, Git, and Subversion tests
are suceptible to the same problem, but are more likely to have their
corresponding command line tools available in standard system paths.

This change reworks the SSH unit tests to perform tool lookups in order
to avoid this issue. Now, when computing whether SSH tests can be
performed for a given setup, it will SSH locally and run which <tool>
commands, caching results. If a tool can't be found, tests will be
skippped, along with a helpful error message aimed at resolving the

This requires opt-in from the test suites. ssh_required_system_exes
can be set to a list of command line tools needed for a successful SSH
test run. All will be checked, and if any are missing, the tests will

This should avoid annoying, hard-to-diagnose test failures going

Altered the system path on login (in ~/.zshenv) to avoid having any
useful directories.

Ran the test suite before this fix and saw errors with the SSH tests.

Ran after the fix and saw skip messages instead.

Added all necessary paths and re-ran. Saw 100% test completion.

Using pytest -rs to verify the presence and content of skip messages.

Skip SSH tests when a system-wide executable is missing.
  1. Ship It!
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-5.0.x (1ed45e0)