Python 芹菜5.0.1，仅使用ForkPoolWorker-31

我发现很奇怪，我的芹菜工人只记录ForkPoolWorker-31，就像它只使用一个处理器处理一样

即使运行top，也表明只有一个处理器非常繁忙，而其他处理器没有那么繁忙

我和你一起吃芹菜

cellery-A my\u服务.cellery\u任务：cellery\u应用程序工作者--loglevel=INFO-n${cellery\u实例}-E

[2020-11-07 00:16:32,677: INFO/MainProcess] celery@grid12 ready.
[2020-11-07 00:16:36,416: WARNING/ForkPoolWorker-31] 19889
[2020-11-07 00:16:36,427: WARNING/ForkPoolWorker-31] 19934
[2020-11-07 00:16:36,427: WARNING/ForkPoolWorker-31] 19882
[2020-11-07 00:16:36,432: WARNING/ForkPoolWorker-31] 20282
[2020-11-07 00:16:36,441: WARNING/ForkPoolWorker-31] 20031
[2020-11-07 00:16:36,446: WARNING/ForkPoolWorker-31] 19884
[2020-11-07 00:16:36,452: WARNING/ForkPoolWorker-31] 20124
[2020-11-07 00:16:36,456: WARNING/ForkPoolWorker-31] 20030
[2020-11-07 00:17:53,313: WARNING/ForkPoolWorker-31] 19897
[2020-11-07 00:17:53,446: INFO/ForkPoolWorker-31] POST Some logs... [status:200 request:11.930s]
[2020-11-07 00:17:54,099: INFO/ForkPoolWorker-31] Some logs...
[2020-11-07 00:17:55,771: INFO/ForkPoolWorker-31] POST Some logs... [status:200 request:15.501s]
[2020-11-07 00:17:56,307: INFO/ForkPoolWorker-31] 

---

这台机器有48个内核，平均使用率最近我遇到了同样的问题，通过添加flag

-O fair

来运行芹菜命令，我能够解决这个问题

我的全部命令如下：

#“-O fair”是预工作人员同时执行任务的关键组件
#芹菜应用程序是我的程序中带有芹菜实例的模块名
#cel_app_worker是芹菜工人的名字
#-P prefork-不是必需的，因为它是默认值，但我决定保留它
芹菜-芹菜应用程序工作者--loglevel=INFO--concurrency=8-O fair-P prefork-n cel\u应用程序工作者

请尝试一下，让我知道它是否适合你

我在Docker中使用芹菜应用程序，Dockerfile：

FROM python:3.7-alpine

WORKDIR /usr/src/app

RUN apk add --no-cache tzdata

ENV TZ=Europe/Moscow

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Create a group and user
RUN addgroup -S appgroup && adduser -S celery_user -G appgroup

# Tell docker that all future commands should run as the appuser user
USER celery_user

# !! "-O fair" is a key component for simultaneous task execution by on worker !!
CMD celery -A celery_app worker --loglevel=INFO --concurrency=8 -O fair -P prefork -n cel_app_worker

FROM python:3.7-alpine

WORKDIR /usr/src/app

RUN apk add --no-cache tzdata

ENV TZ=Europe/Moscow

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Create a group and user
RUN addgroup -S appgroup && adduser -S celery_user -G appgroup

# Tell docker that all future commands should run as the appuser user
USER celery_user

# !! "-O fair" is a key component for simultaneous task execution by on worker !!
CMD celery -A celery_app worker --loglevel=INFO --concurrency=8 -O fair -P prefork -n cel_app_worker