• 
      

    Add initial support for message broker backends.

    Review Request #15196 — Created July 24, 2026 and updated

    Information

    Review Board
    release-9.x

    Reviewers

    This introduces reviewboard.broker, which provides official support in
    Review Board for communicating with message brokers (such as RabbitMQ)
    and sending tasks or broadcasts to any workers connected to the broker.

    This initial change offers base broker backend support, a local
    filesystem-based broker, a broker registry, and the beginnings of broker
    configuration.

    A broker backend is responsible for connecting to a broker service,
    allowing tasks to be sent or messages to be broadcast to all active
    workers, and gathering worker status.

    We're using Celery for the main broker work, since that's pretty
    complete, but the architecture doesn't mandate this. BaseBrokerBackend
    doesn't care about the transport, while BaseCeleryBrokerBackend
    manages all the Celery state (and ties it to the instance, rather than
    registering a global Celery instance).

    Subclasses using Celery simply need to inherit from
    BaseCeleryBrokerBackend and override get_celery_config() to return
    the configuration and broker URI needed.

    There's currently a single built-in filesystem-based broker. It stores
    messages and worker registrations in the site's data directory for local
    workers to access. This is a default that will be usable with a future
    version of Review Bot and with an upcoming local-only companion worker
    responsible for background tasks.

    Future changes will implement worker scanning, more backends,
    configuration, and the local background task companion worker.

    Unit tests pass.

    Tested the basic functionality in combination with other changes and
    a modified Review Bot.

    Summary ID
    Add initial support for message broker backends.
    This introduces `reviewboard.broker`, which provides official support in Review Board for communicating with message brokers (such as RabbitMQ) and sending tasks or broadcasts to any workers connected to the broker. This initial change offers base broker backend support, a local filesystem-based broker, a broker registry, and the beginnings of broker configuration. A broker backend is responsible for connecting to a broker service, allowing tasks to be sent or messages to be broadcast to all active workers, and gathering worker status. We're using Celery for the main broker work, since that's pretty complete, but the architecture doesn't mandate this. `BaseBrokerBackend` doesn't care about the transport, while `BaseCeleryBrokerBackend` manages all the Celery state (and ties it to the instance, rather than registering a global `Celery` instance). Subclasses using Celery simply need to inherit from `BaseCeleryBrokerBackend` and override `get_celery_config()` to return the configuration and broker URI needed. There's currently a single built-in filesystem-based broker. It stores messages and worker registrations in the site's data directory for local workers to access. This is a default that will be usable with a future version of Review Bot and with an upcoming local-only companion worker responsible for background tasks. Future changes will implement worker scanning, more backends, configuration, and the local background task companion worker.
    8cea3ad0b7ef37b01547b9fc82054e2f82591cf3
    Description From Last Updated

    There's no reviewboard/broker/base/__init__.py file (exacerbated by us using namespaces = false in pyproject.toml's tool.setuptools.packages.find)

    david david

    BaseBrokerBackend creates a self.logger but everything in here uses a module-level logger.

    david david

    This logs the message but never resets the value. We'd then hit the assertion below in dev, or just crash …

    david david

    We should probably create with a more locked-down permissions mask.

    david david

    I don't know if you've had a chance to look at the spec I put together on tasks, but I …

    david david

    This doesn't match the implementation/tests. For send_task, the command name is in the headers (message_headers['task']) and the body is [args, …

    david david

    This should probably protect with threading.Lock. It would be nice to also listen to the siteconfig sync (or put in …

    david david

    There's a lot of duplication in here. It might be nice to have a _get_broker_settings(broker_id) helper which can validate and …

    david david

    This is using a module-level logger instead of self.logger. Same for the other one just below.

    david david

    Celery's ctor doesn't define a name arg, but it does take in **kwargs and silently ignore anything it doesn't know …

    david david

    These should match the same order as they appear in the signature.

    david david

    This is passing instance as a positional arg, but the first argument for KeyValueForm.__init__() is data. We need to explicitly …

    david david

    Can we format as: from djblets.registries.registry import ( ALREADY_REGISTERED, ... UNREGISTER, )

    david david

    This is defining the entry point but the class doesn't inherit from EntryPointRegistry. If we fix that we also need …

    david david

    Our other registries use %(item)s. As-is this will render as: "<class 'LocalBrokerBackend'>" is already a registered broker backend. (with both …

    david david

    Same comment about %(item)s vs %(item)r

    david david

    Should be reviewboard.broker.base.backends.BaseBrokerBackend

    david david

    undefined name 'JSONDict' Column: 33 Error code: F821

    reviewbot reviewbot

    It looks like this is using an outdated name for get_celery_config. Same for other tests below.

    david david

    Any time we open a file in text mode we should pass encoding=. When that's present I prefer to also …

    david david
    Checks run (1 failed, 1 succeeded)
    flake8 failed.
    JSHint passed.

    flake8

    david
    1. 
        
    2. Show all issues

      There's no reviewboard/broker/base/__init__.py file (exacerbated by us using namespaces = false in pyproject.toml's tool.setuptools.packages.find)

    3. reviewboard/broker/backends/local.py (Diff revision 1)
       
       
      Show all issues

      BaseBrokerBackend creates a self.logger but everything in here uses a module-level logger.

    4. reviewboard/broker/backends/local.py (Diff revision 1)
       
       
       
       
       
      Show all issues

      This logs the message but never resets the value. We'd then hit the assertion below in dev, or just crash in other ways in prod.

    5. reviewboard/broker/backends/local.py (Diff revision 1)
       
       
       
      Show all issues

      We should probably create with a more locked-down permissions mask.

    6. reviewboard/broker/backends/local.py (Diff revision 1)
       
       
      Show all issues

      I don't know if you've had a chance to look at the spec I put together on tasks, but I don't think we should use a result backend.

      Instead, what I had planned out was a task model that could store (smaller) results directly, and larger things (like doc conversion or review bot) end up mapping to domain objects like reviews or file attachments.

      If we end up actually using a result backend, this would need to be fixed to use "file://localhost{queue_path}" (no extra slash). Celery strips off 16 characters for "file://localhost". For absolute queue paths that currently ends up with a double leading slash, and if somehow the queue path is a relative path, it would end up adding an extra leading slash turning it into an (incorrect) absolute path.

    7. reviewboard/broker/base/backends.py (Diff revision 1)
       
       
       
       
       
       
       
       
       
       
      Show all issues

      This doesn't match the implementation/tests.

      For send_task, the command name is in the headers (message_headers['task']) and the body is [args, kwargs, embed] as per Celery protocol 2.

      For broadcast, the body is a {'method': ..., 'arguments': ...} dict.

    8. reviewboard/broker/base/backends.py (Diff revision 1)
       
       
      Show all issues

      This should probably protect with threading.Lock.

      It would be nice to also listen to the siteconfig sync (or put in a TODO comment about that).

    9. reviewboard/broker/base/backends.py (Diff revision 1)
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
       
      Show all issues

      There's a lot of duplication in here.

      It might be nice to have a _get_broker_settings(broker_id) helper which can validate and return the settings. That would be independently testable, and reusable by BaseBrokerSettingsForm.__init__, which currently does the same lookup but with no validation.

    10. reviewboard/broker/base/backends.py (Diff revision 1)
       
       
      Show all issues

      This is using a module-level logger instead of self.logger. Same for the other one just below.

    11. reviewboard/broker/base/backends.py (Diff revision 1)
       
       
      Show all issues

      Celery's ctor doesn't define a name arg, but it does take in **kwargs and silently ignore anything it doesn't know about.

      I think this was intended to be main='reviewboard'

    12. reviewboard/broker/base/forms.py (Diff revision 1)
       
       
       
       
       
       
       
       
       
      Show all issues

      These should match the same order as they appear in the signature.

    13. reviewboard/broker/base/forms.py (Diff revision 1)
       
       
      Show all issues

      This is passing instance as a positional arg, but the first argument for KeyValueForm.__init__() is data. We need to explicitly pass instance=instance here.

    14. reviewboard/broker/registry.py (Diff revision 1)
       
       
       
       
       
      Show all issues

      Can we format as:

      from djblets.registries.registry import (
          ALREADY_REGISTERED,
          ...
          UNREGISTER,
      )
      
    15. reviewboard/broker/registry.py (Diff revision 1)
       
       
      Show all issues

      This is defining the entry point but the class doesn't inherit from EntryPointRegistry.

      If we fix that we also need a yield from super().get_defaults() inside the get_defaults() implementation.

    16. reviewboard/broker/registry.py (Diff revision 1)
       
       
      Show all issues

      Our other registries use %(item)s. As-is this will render as:

      "<class 'LocalBrokerBackend'>" is already a registered broker backend.

      (with both quotes and repr). Alternatively just remove the quotes.

    17. reviewboard/broker/registry.py (Diff revision 1)
       
       
      Show all issues

      Same comment about %(item)s vs %(item)r

    18. reviewboard/broker/registry.py (Diff revision 1)
       
       
      Show all issues

      Should be reviewboard.broker.base.backends.BaseBrokerBackend

    19. Show all issues

      It looks like this is using an outdated name for get_celery_config. Same for other tests below.

    20. Show all issues

      Any time we open a file in text mode we should pass encoding=. When that's present I prefer to also be explicit with mode='r'.

    21.