Summary

Add distributed lock functionality, and locked cache updates.

Review Request #14628 — Created Oct. 8, 2025 and updated Oct. 9, 2025, 7:38 a.m.

Information

Owner

chipx86

Repository

Djblets

Branch

release-5.x

Bugs

Depends On

Reviewers

Groups

djblets

People

Description

This introduces djblets.protect, a new module for service protection

capabilities, and specifically djblets.protect.locks.CacheLock, which

is a simple distributed lock utilizing the cache backend. This can help

avoid cache stampede issues, and overall reduce the work required by a

service.
Locks have an expiration, and consumers can block waiting on a lock to

be available or return immediately, giving control over how to best

utilize a lock.
Locks are set by performing an atomic add() with a UUID4. If the value

is added, the lock is acquired. If it already exists, the lock has to

either block waiting or return a result. Waiting supports a timeout and

a time between retries.
Locks are released when they expire or (ideally) when release() is

called.
When using a lock as a context manager, both acquiring and releasing the

lock is handled automatically.
The interface is designed to be largely API-compatible with

threading.Lock and similar lock interfaces, but with more flexibility

useful for distributed lock behavior.
A pattern I expect to be common will be to lock a cache key when

calculating state to store and then writing it, which may be expensive

(for instance, talking to a remote service and storing the result).
For this, cache_memoize() and cache_memoize_iter() have been updated

to work with locks. They now take a lock= argument, which accepts a

CacheLock with the parameters controlling the lock behavior. If

provided, the lock will be acquired if the initial fetch doesn't yield a

value. A second fetch is then attempted (in case it had to wait for

another process to finish), and if it still needs to compute data to

cache, it will do so under the protection of the lock, releasing when

complete.
Locks are entirely optional and not enabled by default for any current

caching behavior, but are something we'll likely want to opt into any

time we're working on caching something that's expensive to generate.

Testing Done

Unit tests pass.

Commits

Summary	ID
Add distributed lock functionality, and locked cache updates. This introduces `djblets.protect`, a new module for service protection capabilities, and specifically `djblets.protect.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. The interface is designed to be largely API-compatible with `threading.Lock` and similar lock interfaces, but with more flexibility useful for distributed lock behavior. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.	c181dc802fc56790d92b3494a6790ec809bdfce8

Summary

Add distributed lock functionality, and locked cache updates.

This introduces `djblets.protect`, a new module for service protection capabilities, and specifically `djblets.protect.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. The interface is designed to be largely API-compatible with `threading.Lock` and similar lock interfaces, but with more flexibility useful for distributed lock behavior. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.

c181dc802fc56790d92b3494a6790ec809bdfce8

Issues

Description	From	Last Updated
Can we add debug logging for when locks are acquired, extended, and released? This seems like potentially a cause of …	david	Oct. 9, 2025, 7:38 a.m.
Remove this blank line?	david	Oct. 9, 2025, 7:38 a.m.
typo: as -> was	david	Oct. 9, 2025, 7:38 a.m.
Do we want to add any validation to these (ex: no negative numbers, timeout should be longer than retry, etc)?	david	Oct. 9, 2025, 7:38 a.m.
These attributes are actually named timeout_secs and retry_secs	david	Oct. 9, 2025, 7:38 a.m.
Can we add the lock key in this exception message? AssertionError also seems like maybe not the best type. Perhaps …	david	Oct. 9, 2025, 7:38 a.m.
Comparing a float like this isn't reliable. While we could use something like math.isclose(), I think a much better option …	david	Oct. 9, 2025, 7:38 a.m.
Seems like we likely want >= instead of >	david	Oct. 9, 2025, 7:38 a.m.
The implementation here doesn't return anything	david	Oct. 9, 2025, 7:38 a.m.
typo: released -> release	david	Oct. 9, 2025, 7:38 a.m.
It seems like there's a potential race here where in between the check and the delete, the existing lock could …	david	Oct. 9, 2025, 7:38 a.m.

flake8 passed.

JSHint passed.

Change Summary:


Moved to a new djblets.protect, which will be the place for other service protection code, like rate limiting.
Added to the README and codebase docs.

Description:

~		This introduces `djblets.cache.locks.CacheLock`, which is a simple
~		distributed lock utilizing the cache backend. This can help avoid cache
~		stampede issues, and overall reduce the work required by a service.
	~	This introduces `djblets.protect`, a new module for service protection
	~	capabilities, and specifically `djblets.protect.locks.CacheLock`, which
	~	is a simple distributed lock utilizing the cache backend. This can help
	+	avoid cache stampede issues, and overall reduce the work required by a
	+	service.

		Locks have an expiration, and consumers can block waiting on a lock to
		be available or return immediately, giving control over how to best
		utilize a lock.

		Locks are set by performing an atomic `add()` with a UUID4. If the value
		is added, the lock is acquired. If it already exists, the lock has to
		either block waiting or return a result. Waiting supports a timeout and
		a time between retries.

		Locks are released when they expire or (ideally) when `release()` is
		called.

		When using a lock as a context manager, both acquiring and releasing the
		lock is handled automatically.

		A pattern I expect to be common will be to lock a cache key when
		calculating state to store and then writing it, which may be expensive
		(for instance, talking to a remote service and storing the result).

		For this, `cache_memoize()` and `cache_memoize_iter()` have been updated
		to work with locks. They now take a `lock=` argument, which accepts a
		`CacheLock` with the parameters controlling the lock behavior. If
		provided, the lock will be acquired if the initial fetch doesn't yield a
		value. A second fetch is then attempted (in case it had to wait for
		another process to finish), and if it still needs to compute data to
		cache, it will do so under the protection of the lock, releasing when
		complete.

		Locks are entirely optional and not enabled by default for any current
		caching behavior, but are something we'll likely want to opt into any
		time we're working on caching something that's expensive to generate.

Commits:

	Summary	ID
	Add distributed lock functionality, and locked cache updates. This introduces `djblets.cache.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.	a0bda35677806af82016bd478fffdf63c61b6267
	Add distributed lock functionality, and locked cache updates. This introduces `djblets.protect`, a new module for service protection capabilities, and specifically `djblets.protect.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.	5ad002a1428c8244e6a2e92d94a3e93f8d952c50

Diff:

Revision 2 (+2652 -240)

Show changes

	README.rst
	djblets/cache/backend.py
	djblets/cache/tests/test_backend.py
	djblets/protect/__init__.py
	djblets/protect/locks.py
	djblets/protect/tests/test_cache_lock.py
	docs/djblets/coderef/index.rst

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Change Summary:

Reworked some of the API to be compatible with threading.Lock and similar.

Description:

		This introduces `djblets.protect`, a new module for service protection
		capabilities, and specifically `djblets.protect.locks.CacheLock`, which
		is a simple distributed lock utilizing the cache backend. This can help
		avoid cache stampede issues, and overall reduce the work required by a
		service.

		Locks have an expiration, and consumers can block waiting on a lock to
		be available or return immediately, giving control over how to best
		utilize a lock.

		Locks are set by performing an atomic `add()` with a UUID4. If the value
		is added, the lock is acquired. If it already exists, the lock has to
		either block waiting or return a result. Waiting supports a timeout and
		a time between retries.

		Locks are released when they expire or (ideally) when `release()` is
		called.

		When using a lock as a context manager, both acquiring and releasing the
		lock is handled automatically.

	+	The interface is designed to be largely API-compatible with
	+	`threading.Lock` and similar lock interfaces, but with more flexibility
	+	useful for distributed lock behavior.
	+
		A pattern I expect to be common will be to lock a cache key when
		calculating state to store and then writing it, which may be expensive
		(for instance, talking to a remote service and storing the result).

		For this, `cache_memoize()` and `cache_memoize_iter()` have been updated
		to work with locks. They now take a `lock=` argument, which accepts a
		`CacheLock` with the parameters controlling the lock behavior. If
		provided, the lock will be acquired if the initial fetch doesn't yield a
		value. A second fetch is then attempted (in case it had to wait for
		another process to finish), and if it still needs to compute data to
		cache, it will do so under the protection of the lock, releasing when
		complete.

		Locks are entirely optional and not enabled by default for any current
		caching behavior, but are something we'll likely want to opt into any
		time we're working on caching something that's expensive to generate.

Commits:

	Summary	ID
	Add distributed lock functionality, and locked cache updates. This introduces `djblets.protect`, a new module for service protection capabilities, and specifically `djblets.protect.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.	5ad002a1428c8244e6a2e92d94a3e93f8d952c50
	Add distributed lock functionality, and locked cache updates. This introduces `djblets.protect`, a new module for service protection capabilities, and specifically `djblets.protect.locks.CacheLock`, which is a simple distributed lock utilizing the cache backend. This can help avoid cache stampede issues, and overall reduce the work required by a service. Locks have an expiration, and consumers can block waiting on a lock to be available or return immediately, giving control over how to best utilize a lock. Locks are set by performing an atomic `add()` with a UUID4. If the value is added, the lock is acquired. If it already exists, the lock has to either block waiting or return a result. Waiting supports a timeout and a time between retries. Locks are released when they expire or (ideally) when `release()` is called. When using a lock as a context manager, both acquiring and releasing the lock is handled automatically. The interface is designed to be largely API-compatible with `threading.Lock` and similar lock interfaces, but with more flexibility useful for distributed lock behavior. A pattern I expect to be common will be to lock a cache key when calculating state to store and then writing it, which may be expensive (for instance, talking to a remote service and storing the result). For this, `cache_memoize()` and `cache_memoize_iter()` have been updated to work with locks. They now take a `lock=` argument, which accepts a `CacheLock` with the parameters controlling the lock behavior. If provided, the lock will be acquired if the initial fetch doesn't yield a value. A second fetch is then attempted (in case it had to wait for another process to finish), and if it still needs to compute data to cache, it will do so under the protection of the lock, releasing when complete. Locks are entirely optional and not enabled by default for any current caching behavior, but are something we'll likely want to opt into any time we're working on caching something that's expensive to generate.	c181dc802fc56790d92b3494a6790ec809bdfce8

Diff:

Revision 3 (+2764 -240)

Show changes

	README.rst
	djblets/cache/backend.py
	djblets/cache/tests/test_backend.py
	djblets/protect/__init__.py
	djblets/protect/locks.py
	djblets/protect/tests/test_cache_lock.py
	docs/djblets/coderef/index.rst

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Fix it!

An issue was opened. Show all issues

Can we add debug logging for when locks are acquired, extended, and released? This seems like potentially a cause of difficult bugs.

djblets/cache/backend.py (Diff revision 3)
An issue was opened. Show all issues
```
Remove this blank line?
```
djblets/cache/backend.py (Diff revision 3)
An issue was opened. Show all issues
```
typo: as -> was
```

djblets/protect/locks.py (Diff revision 3)

An issue was opened. Show all issues

Do we want to add any validation to these (ex: no negative numbers, timeout should be longer than retry, etc)?

djblets/protect/locks.py (Diff revision 3)

An issue was opened. Show all issues

These attributes are actually named timeout_secs and retry_secs

djblets/protect/locks.py (Diff revision 3)

An issue was opened. Show all issues

Can we add the lock key in this exception message?

AssertionError also seems like maybe not the best type. Perhaps just a RuntimeError?

djblets/protect/locks.py (Diff revision 3)

An issue was opened. Show all issues

Comparing a float like this isn't reliable. While we could use something like math.isclose(), I think a much better option than a -1 magic value would be to define a sentinel value to use to indicate no timeout.

djblets/protect/locks.py (Diff revision 3)
An issue was opened. Show all issues
```
Seems like we likely want >= instead of >
```
djblets/protect/locks.py (Diff revision 3)
An issue was opened. Show all issues
```
The implementation here doesn't return anything
```
djblets/protect/locks.py (Diff revision 3)
An issue was opened. Show all issues
```
typo: released -> release
```

djblets/protect/locks.py (Diff revision 3)

An issue was opened. Show all issues

It seems like there's a potential race here where in between the check and the delete, the existing lock could timeout and be acquired by another user. I don't suppose there are any atomic operations we could do for this?