Update Review Bot 3 to use Celery 5 on Python 3.

Review Request #12157 — Created March 16, 2022 and submitted

Information

ReviewBot
release-3.0.x

Reviewers

Review Bot 2 was using Celery 4, which is no longer maintained, and
Review Bot 3 inherited that dependency. Between Celery 4 (which is no
longer maintained and has security problems) and 5, much has changed
when it comes to the celery worker command setup and invocation, and
this is what we were wrapping in order to start a process. This is no
longer feasible.

Now, we construct the Worker object ourselves. This is more
future-proof, though comes with a few requirements on our end (some
patching functions we have to call, and some arguments to process).

We now support setting up both Celery 3 (on Python 2.7) and Celery 5 (on
Python 3). This code is confined to celery.py, allowing main.py to
be nice and simple. In time, this will also help us upgrade to newer
versions of Celery or to switch to another backend (perhaps
experimentally).

Tested running on Python 2.7 and 3.x in both detached and in-process
modes, with various combinations of options.

Unit tests pass.

Summary ID
Update Review Bot 3 to use Celery 5 on Python 3.
Review Bot 2 was using Celery 4, which is no longer maintained, and Review Bot 3 inherited that dependency. Between Celery 4 (which is no longer maintained and has security problems) and 5, much has changed when it comes to the `celery worker` command setup and invocation, and this is what we were wrapping in order to start a process. This is no longer feasible. Now, we construct the `Worker` object ourselves. This is more future-proof, though comes with a few requirements on our end (some patching functions we have to call, and some arguments to process). We now support setting up both Celery 3 (on Python 2.7) and Celery 5 (on Python 3). This code is confined to `celery.py`, allowing `main.py` to be nice and simple. In time, this will also help us upgrade to newer versions of Celery or to switch to another backend (perhaps experimentally).
a3e04deef03a172651fd2dc068b7b3f52e2fb57a
Description From Last Updated

F401 'celery.__version__ as celery_version_str' imported but unused

reviewbotreviewbot

F811 redefinition of unused 'worker' from line 13

reviewbotreviewbot

F401 'textwrap' imported but unused

reviewbotreviewbot
Checks run (1 failed, 1 succeeded)
flake8 failed.
JSHint passed.

flake8

chipx86
Review request changed

Change Summary:

  • Removed two unused imports.

Commits:

Summary ID
Update Review Bot 3 to use Celery 5 on Python 3.
Review Bot 2 was using Celery 4, which is no longer maintained, and Review Bot 3 inherited that dependency. Between Celery 4 (which is no longer maintained and has security problems) and 5, much has changed when it comes to the `celery worker` command setup and invocation, and this is what we were wrapping in order to start a process. This is no longer feasible. Now, we construct the `Worker` object ourselves. This is more future-proof, though comes with a few requirements on our end (some patching functions we have to call, and some arguments to process). We now support setting up both Celery 3 (on Python 2.7) and Celery 5 (on Python 3). This code is confined to `celery.py`, allowing `main.py` to be nice and simple. In time, this will also help us upgrade to newer versions of Celery or to switch to another backend (perhaps experimentally).
53357259e1d2433f08c86f79e7c442910c8fbae7
Update Review Bot 3 to use Celery 5 on Python 3.
Review Bot 2 was using Celery 4, which is no longer maintained, and Review Bot 3 inherited that dependency. Between Celery 4 (which is no longer maintained and has security problems) and 5, much has changed when it comes to the `celery worker` command setup and invocation, and this is what we were wrapping in order to start a process. This is no longer feasible. Now, we construct the `Worker` object ourselves. This is more future-proof, though comes with a few requirements on our end (some patching functions we have to call, and some arguments to process). We now support setting up both Celery 3 (on Python 2.7) and Celery 5 (on Python 3). This code is confined to `celery.py`, allowing `main.py` to be nice and simple. In time, this will also help us upgrade to newer versions of Celery or to switch to another backend (perhaps experimentally).
54a9e4e91e1372276b79cbbe6dd1325bac998660

Diff:

Revision 2 (+318 -158)

Show changes

Checks run (1 failed, 1 succeeded)

flake8 failed.
JSHint passed.

flake8

chipx86
chipx86
david
  1. Ship It!
  2. 
      
chipx86
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-3.0.x (87c0366)
Loading...