• 
      

    Change Celery initialization to ensure logging is set up.

    Review Request #11512 — Created March 11, 2021 and submitted

    Information

    ReviewBot
    release-3.0.x

    Reviewers

    Celery expects to manage pretty much everything, including logging. It
    sets up logging handlers, formatting, etc., and it does this very
    shortly before it begins registering queues and displaying the initial
    sort of splash screen that lists them.

    Any code that attempts to register a logger prior to this will fail, and
    any top-level log statements being executed (including our dependency
    warnings for tools) will use the default logger with the default
    formatting and not acknowledge any filters or log files passed to
    Celery.

    While not terribly well-documented, Celery does support a number of
    signal handlers that allow for performing operations during setup. We
    now listen to one of these, celeryd_after_setup, and perform our own
    configuration loading and dependency checks here.

    We also set the queues here, which we can do by setting a property. This
    is supported in the public API (intended for dynamic queue computation),
    and is equivalent to what happens when setting the configuration
    up-front.

    Tested launching Review Bot on both Python 2.7 (with Celery 3.x) and
    Python 3 (with Celery 4.x). Verified in both cases that logging was
    properly set up, logging statements from dependency checks and upcoming
    code was formatted properly and reacted to Celery log configuration,
    and that queues for tools were properly registered.

    Summary ID
    Change Celery initialization to ensure logging is set up.
    Celery expects to manage pretty much everything, including logging. It sets up logging handlers, formatting, etc., and it does this very shortly before it begins registering queues and displaying the initial sort of splash screen that lists them. Any code that attempts to register a logger prior to this will fail, and any top-level log statements being executed (including our dependency warnings for tools) will use the default logger with the default formatting and not acknowledge any filters or log files passed to Celery. While not terribly well-documented, Celery does support a number of signal handlers that allow for performing operations during setup. We now listen to one of these, `celeryd_after_setup`, and perform our own configuration loading and dependency checks here. We also set the queues here, which we can do by setting a property. This is supported in the public API (intended for dynamic queue computation), and is equivalent to what happens when setting the configuration up-front.
    2f0b0cd4a7ecaa2f33cbcd3a444278cf57904c19
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-3.0.x (8c8e015)