Django 在弹性豆茎上运行芹菜工人

我试着把芹菜放在弹性豆茎上，以执行定期任务。除了弹性豆茎上的芹菜工人外，我什么都做了。它部署得很好，并且正在注册任务，但当我在Cellery.py或通过django Cellery beat设置周期性任务时，它不会执行。我从EB中取出原木，然后

/opt/python/log/supervisord.log
-------------------------------------
2019-09-20 16:32:50,831 INFO spawned: 'celeryd-worker' with pid 712
2019-09-20 16:32:53,929 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:32:54,930 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 16:41:43,327 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 16:41:44,333 INFO spawned: 'celeryd-beat' with pid 1664
2019-09-20 16:41:54,442 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 16:41:54,689 INFO spawned: 'celeryd-worker' with pid 1670
2019-09-20 16:41:57,844 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:41:58,847 INFO spawned: 'celeryd-worker' with pid 1676
2019-09-20 16:42:02,037 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:04,045 INFO spawned: 'celeryd-worker' with pid 1711
2019-09-20 16:42:05,271 INFO stopped: httpd (exit status 0)
2019-09-20 16:42:05,275 INFO spawned: 'httpd' with pid 1717
2019-09-20 16:42:07,241 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-20 16:42:07,898 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:11,041 INFO spawned: 'celeryd-worker' with pid 1841
2019-09-20 16:42:11,344 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 16:42:12,349 INFO spawned: 'celeryd-beat' with pid 1845
2019-09-20 16:42:14,527 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:15,529 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 16:42:22,426 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 16:42:22,673 INFO spawned: 'celeryd-worker' with pid 1852
2019-09-20 16:42:25,760 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:26,763 INFO spawned: 'celeryd-worker' with pid 1857
2019-09-20 16:42:29,890 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:31,897 INFO spawned: 'celeryd-worker' with pid 1872
2019-09-20 16:42:35,010 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:38,360 INFO spawned: 'celeryd-worker' with pid 1879
2019-09-20 16:42:41,484 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 16:42:42,485 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 17:01:45,639 INFO spawned: 'celeryd-worker' with pid 3075
2019-09-20 17:01:46,392 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 17:01:47,397 INFO spawned: 'celeryd-beat' with pid 3080
2019-09-20 17:01:47,924 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:01:49,695 INFO spawned: 'celeryd-worker' with pid 3085
2019-09-20 17:01:51,339 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:01:53,343 INFO spawned: 'celeryd-worker' with pid 3089
2019-09-20 17:01:54,890 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:01:57,894 INFO spawned: 'celeryd-worker' with pid 3094
2019-09-20 17:01:57,895 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 17:01:58,386 INFO stopped: celeryd-worker (terminated by SIGTERM)
2019-09-20 17:01:59,391 INFO spawned: 'celeryd-worker' with pid 3099
2019-09-20 17:02:00,957 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:02:01,958 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 17:02:03,508 INFO stopped: httpd (exit status 0)
2019-09-20 17:02:03,513 INFO spawned: 'httpd' with pid 3144
2019-09-20 17:02:04,678 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-20 17:02:08,612 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 17:02:09,617 INFO spawned: 'celeryd-beat' with pid 3268
2019-09-20 17:02:20,058 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 17:02:20,306 INFO spawned: 'celeryd-worker' with pid 3274
2019-09-20 17:02:21,860 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:02:22,863 INFO spawned: 'celeryd-worker' with pid 3278
2019-09-20 17:02:24,429 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:02:26,437 INFO spawned: 'celeryd-worker' with pid 3291
2019-09-20 17:02:27,995 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:02:31,970 INFO spawned: 'celeryd-worker' with pid 3296
2019-09-20 17:02:33,535 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:02:34,537 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 17:17:05,637 INFO spawned: 'celeryd-worker' with pid 4245
2019-09-20 17:17:07,195 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:17:08,197 INFO spawned: 'celeryd-worker' with pid 4250
2019-09-20 17:17:09,750 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:17:11,757 INFO spawned: 'celeryd-worker' with pid 4268
2019-09-20 17:17:12,996 INFO stopped: httpd (exit status 0)
2019-09-20 17:17:13,667 INFO spawned: 'httpd' with pid 4274
2019-09-20 17:17:14,000 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:17:15,001 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-20 17:17:17,749 INFO spawned: 'celeryd-worker' with pid 4356
2019-09-20 17:17:19,939 INFO stopped: celeryd-worker (terminated by SIGTERM)
2019-09-20 17:17:20,944 INFO spawned: 'celeryd-worker' with pid 4400
2019-09-20 17:17:22,486 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:17:23,005 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 17:28:31,137 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 17:28:32,142 INFO spawned: 'celeryd-beat' with pid 5121
2019-09-20 17:28:42,799 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 17:28:43,049 INFO spawned: 'celeryd-worker' with pid 5127
2019-09-20 17:28:44,655 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:28:45,745 INFO spawned: 'celeryd-worker' with pid 5145
2019-09-20 17:28:46,959 INFO stopped: httpd (exit status 0)
2019-09-20 17:28:46,963 INFO spawned: 'httpd' with pid 5151
2019-09-20 17:28:47,955 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:28:48,957 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-20 17:28:49,961 INFO spawned: 'celeryd-worker' with pid 5221
2019-09-20 17:28:52,555 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:28:53,967 INFO stopped: celeryd-beat (exit status 0)
2019-09-20 17:28:54,972 INFO spawned: 'celeryd-beat' with pid 5278
2019-09-20 17:28:55,975 INFO spawned: 'celeryd-worker' with pid 5282
2019-09-20 17:28:57,801 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:28:58,803 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly
2019-09-20 17:29:05,811 INFO success: celeryd-beat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2019-09-20 17:29:06,058 INFO spawned: 'celeryd-worker' with pid 5304
2019-09-20 17:29:07,616 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:29:08,618 INFO spawned: 'celeryd-worker' with pid 5309
2019-09-20 17:29:10,142 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:29:12,149 INFO spawned: 'celeryd-worker' with pid 5322
2019-09-20 17:29:13,708 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:29:17,450 INFO spawned: 'celeryd-worker' with pid 5326
2019-09-20 17:29:18,998 INFO exited: celeryd-worker (exit status 1; not expected)
2019-09-20 17:29:20,000 INFO gave up: celeryd-worker entered FATAL state, too many start retries too quickly

芹菜工人似乎正在启动/重新启动，所以问题一定是芹菜工人？我在这方面非常新手，所以我不确定问题出在哪里

.ebextensions/files/cellery_configuration.txt

#!/usr/bin/env bash

# Get django environment variables
celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g' | sed 's/%/%%/g'`
celeryenv=${celeryenv%?}

# Create celery configuraiton script
celeryconf="[program:celeryd-worker]
; Set full path to celery program if using virtualenv
command=/opt/python/run/venv/bin/celery worker -A backend -P solo --loglevel=INFO -n worker.%%h

directory=/opt/python/current/app
user=nobody
numprocs=1
stdout_logfile=/var/log/celery-worker.log
stderr_logfile=/var/log/celery-worker.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true

; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998

environment=$celeryenv

[program:celeryd-beat]
; Set full path to celery program if using virtualenv
command=/opt/python/run/venv/bin/celery beat -A backend --loglevel=INFO --workdir=/tmp -S django --pidfile /tmp/celerybeat.pid

directory=/opt/python/current/app
user=nobody
numprocs=1
stdout_logfile=/var/log/celery-beat.log
stderr_logfile=/var/log/celery-beat.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; When resorting to send SIGKILL to the program to terminate it
; send SIGKILL to its whole process group instead,
; taking care of its children as well.
killasgroup=true

; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998

environment=$celeryenv"

# Create the celery supervisord conf script
echo "$celeryconf" | tee /opt/python/etc/celery.conf

# Add configuration script to supervisord conf (if not there already)
if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
  then
  echo "[include]" | tee -a /opt/python/etc/supervisord.conf
  echo "files: celery.conf" | tee -a /opt/python/etc/supervisord.conf
fi

# Reread the supervisord config
supervisorctl -c /opt/python/etc/supervisord.conf reread

# Update supervisord in cache without restarting all services
supervisorctl -c /opt/python/etc/supervisord.conf update

# Start/Restart celeryd through supervisord
supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd-beat
supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd-worker

02-python.config

packages:
  yum:
    libcurl-devel: []


container_commands:
  01_upgrade_pip_for_venv:
    command: "/opt/python/run/venv/bin/pip install --upgrade pip"
  04_celery_tasks:
    command: "cat .ebextensions/files/celery_configuration.txt > /opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh && chmod 744 /opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh"
    leader_only: true
  05_celery_tasks_run:
    command: "/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh"
    leader_only: true

芹菜

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from celery.schedules import crontab

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'backend.settings')

app = Celery('backend')

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()


app.conf.beat_schedule = {
    'task_every_5_seconds': {  #name of the scheduler
        'task': 'add_new_task',  # task name which we have created in tasks.py
        'schedule': 10.0,   # set the period of running
    },
}

---

以下是一些需要检查的事项：

• 如果使用芹菜v4，芹菜节拍的start命令看起来是错误的。根据，您应该使用以下内容：

/opt/python/run/venv/bin/cellery beat-A backend--loglevel=INFO--workdir=/tmp--scheduler-django_cellery_beat.scheduler:DatabaseScheduler--pidfile/tmp/celerybeat.pid

• 确保在所有beanstalk实例上只运行一个beat进程（即，每个代理URL一个beat实例）