Change Celery initialization to ensure logging is set up.

Review Request #11512 — Created March 11, 2021 and submitted

Information

ReviewBot
release-3.0.x

Reviewers

Celery expects to manage pretty much everything, including logging. It
sets up logging handlers, formatting, etc., and it does this very
shortly before it begins registering queues and displaying the initial
sort of splash screen that lists them.

Any code that attempts to register a logger prior to this will fail, and
any top-level log statements being executed (including our dependency
warnings for tools) will use the default logger with the default
formatting and not acknowledge any filters or log files passed to
Celery.

While not terribly well-documented, Celery does support a number of
signal handlers that allow for performing operations during setup. We
now listen to one of these, celeryd_after_setup, and perform our own
configuration loading and dependency checks here.

We also set the queues here, which we can do by setting a property. This
is supported in the public API (intended for dynamic queue computation),
and is equivalent to what happens when setting the configuration
up-front.

Tested launching Review Bot on both Python 2.7 (with Celery 3.x) and
Python 3 (with Celery 4.x). Verified in both cases that logging was
properly set up, logging statements from dependency checks and upcoming
code was formatted properly and reacted to Celery log configuration,
and that queues for tools were properly registered.

Summary ID
Change Celery initialization to ensure logging is set up.
Celery expects to manage pretty much everything, including logging. It sets up logging handlers, formatting, etc., and it does this very shortly before it begins registering queues and displaying the initial sort of splash screen that lists them. Any code that attempts to register a logger prior to this will fail, and any top-level log statements being executed (including our dependency warnings for tools) will use the default logger with the default formatting and not acknowledge any filters or log files passed to Celery. While not terribly well-documented, Celery does support a number of signal handlers that allow for performing operations during setup. We now listen to one of these, `celeryd_after_setup`, and perform our own configuration loading and dependency checks here. We also set the queues here, which we can do by setting a property. This is supported in the public API (intended for dynamic queue computation), and is equivalent to what happens when setting the configuration up-front.
2f0b0cd4a7ecaa2f33cbcd3a444278cf57904c19
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-3.0.x (8c8e015)