Fix caching issues with iter_exes_in_path().

Review Request #12641 — Created Sept. 25, 2022 and submitted — Latest diff uploaded

Information

RBTools
release-4.x

Reviewers

iter_exes_in_path() was decorated with @lru_cache in an attempt to
avoid numerous filesystem lookups (particularly in unit tests). This
doesn't actually work, though, as you can't cache a generator. Switching
to pre-populating, caching, and returning a list of items would solve
this, but wouldn't be ideal for the common case of just wanting the
first match.

Instead, this change adds more fine-grained caching within the function.
We now cache the existence of an item per-path. This is built up
incrementally, so if we only ever hit one item in the path before
finding a result, we only need to cache and return that item. If we fail
to find a result from any items in the path, a second attempt will
discover that without any file existence checks.

This avoids sporadic problems with unit test failures.

All unit tests pass.

Diff Revision 1 (Latest)

Commits

First Last Summary ID Author
Fix caching issues with iter_exes_in_path().
`iter_exes_in_path()` was decorated with `@lru_cache` in an attempt to avoid numerous filesystem lookups (particularly in unit tests). This doesn't actually work, though, as you can't cache a generator. Switching to pre-populating, caching, and returning a list of items would solve this, but wouldn't be ideal for the common case of just wanting the first match. Instead, this change adds more fine-grained caching within the function. We now cache the existence of an item per-path. This is built up incrementally, so if we only ever hit one item in the path before finding a result, we only need to cache and return that item. If we fail to find a result from any items in the path, a second attempt will discover that without any file existence checks. This avoids sporadic problems with unit test failures.
ba3cb67f75b3628e818fc8ccdcf0a0ac99cce2af Christian Hammond
rbtools/diffs/tests/test_gnu_diff_tool.py
rbtools/utils/filesystem.py
Loading...