Django/Cellery/Kombu工作者错误：收到并删除了未知消息。错误的目的地？

似乎消息没有正确地放入队列

我将Django与芹菜和Kombu一起使用，以利用Django自己的数据库作为代理后端。我只需要一个非常简单的发布/订阅设置。它最终将部署到Heroku，所以我使用foreman在本地运行。以下是相关代码和信息：

pip冻结

Django==1.4.2
celery==3.0.15
django-celery==3.0.11
kombu==2.5.6

web: source bin/activate; python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 4; python manage.py syncdb
celeryd: python manage.py celeryd -E -B --loglevel=INFO

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  queue.put(id)
  queue.close()

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  message = {
    'task': 'process-next-task',
    'id': str(uuid.uuid4()),
    'args': [id],
    "kwargs": {},
    "retries": 0,
    "eta": str(datetime.datetime.now())
  }
  queue.put(message)
  queue.close()

Procfile

Django==1.4.2
celery==3.0.15
django-celery==3.0.11
kombu==2.5.6

web: source bin/activate; python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 4; python manage.py syncdb
celeryd: python manage.py celeryd -E -B --loglevel=INFO

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  queue.put(id)
  queue.close()

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  message = {
    'task': 'process-next-task',
    'id': str(uuid.uuid4()),
    'args': [id],
    "kwargs": {},
    "retries": 0,
    "eta": str(datetime.datetime.now())
  }
  queue.put(message)
  queue.close()

设置.py

# Celery configuration
import djcelery
CELERY_IMPORTS = ("api.tasks",)
BROKER_URL = "django://localhost//"
djcelery.setup_loader()

@task()
def process_next_task():
  with Connection(settings.BROKER_URL) as conn:
    queue = conn.SimpleQueue('celery')
    message = queue.get(block=True, timeout=1)
    id = int(message.payload)
    try:
      Model.objects.get(id=id)
    except Model.DoesNotExist:
      message.reject()
    else:
      # Do stuff here
      message.ack()
    queue.close()

@task(serializer='json', name='process-next-task')
def process_next_task(id):
  try:
    Model.objects.get(id=int(id))
  except Model.DoesNotExist:
    pass
  else:
    # Do stuff here

放置消息

Django==1.4.2
celery==3.0.15
django-celery==3.0.11
kombu==2.5.6

web: source bin/activate; python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 4; python manage.py syncdb
celeryd: python manage.py celeryd -E -B --loglevel=INFO

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  queue.put(id)
  queue.close()

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  message = {
    'task': 'process-next-task',
    'id': str(uuid.uuid4()),
    'args': [id],
    "kwargs": {},
    "retries": 0,
    "eta": str(datetime.datetime.now())
  }
  queue.put(message)
  queue.close()

api/tasks.py

# Celery configuration
import djcelery
CELERY_IMPORTS = ("api.tasks",)
BROKER_URL = "django://localhost//"
djcelery.setup_loader()

@task()
def process_next_task():
  with Connection(settings.BROKER_URL) as conn:
    queue = conn.SimpleQueue('celery')
    message = queue.get(block=True, timeout=1)
    id = int(message.payload)
    try:
      Model.objects.get(id=id)
    except Model.DoesNotExist:
      message.reject()
    else:
      # Do stuff here
      message.ack()
    queue.close()

@task(serializer='json', name='process-next-task')
def process_next_task(id):
  try:
    Model.objects.get(id=int(id))
  except Model.DoesNotExist:
    pass
  else:
    # Do stuff here

在终端中，

工头启动

工作正常，显示如下：

started with pid 31835
17:08:22 celeryd.1 | started with pid 31836
17:08:22 web.1     | /usr/local/foreman/bin/foreman-runner: line 41: exec: source: not found
17:08:22 web.1     | 2013-02-14 17:08:22 [31838] [INFO] Starting gunicorn 0.16.1
17:08:22 web.1     | 2013-02-14 17:08:22 [31838] [INFO] Listening at: http://0.0.0.0:5000 (31838)
17:08:22 web.1     | 2013-02-14 17:08:22 [31838] [INFO] Using worker: sync
17:08:22 web.1     | 2013-02-14 17:08:22 [31843] [INFO] Booting worker with pid: 31843
17:08:22 web.1     | 2013-02-14 17:08:22 [31844] [INFO] Booting worker with pid: 31844
17:08:22 web.1     | 2013-02-14 17:08:22 [31845] [INFO] Booting worker with pid: 31845
17:08:22 web.1     | 2013-02-14 17:08:22 [31846] [INFO] Booting worker with pid: 31846
17:08:22 celeryd.1 | [2013-02-14 17:08:22,858: INFO/Beat] Celerybeat: Starting...
17:08:22 celeryd.1 | [2013-02-14 17:08:22,870: WARNING/MainProcess] celery@myhost.local ready.
17:08:22 celeryd.1 | [2013-02-14 17:08:22,873: INFO/MainProcess] consumer: Connected to django://localhost//.
17:08:42 celeryd.1 | [2013-02-14 17:08:42,926: WARNING/MainProcess] Received and deleted unknown message. Wrong destination?!?
17:08:42 celeryd.1 | The full contents of the message body was: body: 25 (2b) {content_type:u'application/json' content_encoding:u'utf-8' delivery_info:{u'priority': 0, u'routing_key': u'celery', u'exchange': u'celery'}}

最后两行不会立即显示，但当我的API收到一个POST请求时会显示出来，该请求运行上面put_message部分中的代码。我曾经尝试过使用Kombu的完全成熟的生产者和消费者类，结果是一样的

Kombu的SimpleQueue示例：
芹菜：

有什么想法吗

已编辑

在procfile中更改为

--loglevel=DEBUG

，会将终端输出更改为以下内容：

08:54:33 celeryd.1 | started with pid 555
08:54:33 web.1     | started with pid 554
08:54:33 web.1     | /usr/local/foreman/bin/foreman-runner: line 41: exec: source: not found
08:54:36 web.1     | 2013-02-15 08:54:36 [557] [INFO] Starting gunicorn 0.16.1
08:54:36 web.1     | 2013-02-15 08:54:36 [557] [INFO] Listening at: http://0.0.0.0:5000 (557)
08:54:36 web.1     | 2013-02-15 08:54:36 [557] [INFO] Using worker: sync
08:54:36 web.1     | 2013-02-15 08:54:36 [564] [INFO] Booting worker with pid: 564
08:54:36 web.1     | 2013-02-15 08:54:36 [565] [INFO] Booting worker with pid: 565
08:54:36 web.1     | 2013-02-15 08:54:36 [566] [INFO] Booting worker with pid: 566
08:54:36 web.1     | 2013-02-15 08:54:36 [567] [INFO] Booting worker with pid: 567
08:54:37 celeryd.1 | [2013-02-15 08:54:37,480: DEBUG/MainProcess] [Worker] Loading modules.
08:54:37 celeryd.1 | [2013-02-15 08:54:37,484: DEBUG/MainProcess] [Worker] Claiming components.
08:54:37 celeryd.1 | [2013-02-15 08:54:37,484: DEBUG/MainProcess] [Worker] Building boot step graph.
08:54:37 celeryd.1 | [2013-02-15 08:54:37,484: DEBUG/MainProcess] [Worker] New boot order: {ev, queues, beat, pool, mediator, autoreloader, timers, state-db, autoscaler, consumer}
08:54:37 celeryd.1 | [2013-02-15 08:54:37,489: DEBUG/MainProcess] Starting celery.beat._Process...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,490: DEBUG/MainProcess] celery.beat._Process OK!
08:54:37 celeryd.1 | [2013-02-15 08:54:37,491: DEBUG/MainProcess] Starting celery.concurrency.processes.TaskPool...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,491: INFO/Beat] Celerybeat: Starting...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,506: DEBUG/MainProcess] celery.concurrency.processes.TaskPool OK!
08:54:37 celeryd.1 | [2013-02-15 08:54:37,507: DEBUG/MainProcess] Starting celery.worker.mediator.Mediator...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,507: DEBUG/MainProcess] celery.worker.mediator.Mediator OK!
08:54:37 celeryd.1 | [2013-02-15 08:54:37,507: DEBUG/MainProcess] Starting celery.worker.consumer.BlockingConsumer...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,508: WARNING/MainProcess] celery@myhost.local ready.
08:54:37 celeryd.1 | [2013-02-15 08:54:37,508: DEBUG/MainProcess] consumer: Re-establishing connection to the broker...
08:54:37 celeryd.1 | [2013-02-15 08:54:37,510: INFO/MainProcess] consumer: Connected to django://localhost//.
08:54:37 celeryd.1 | [2013-02-15 08:54:37,628: DEBUG/Beat] Current schedule:
08:54:37 celeryd.1 | <Entry: celery.backend_cleanup celery.backend_cleanup() {<crontab: * 4 * * * (m/h/d/dM/MY)>}
08:54:37 celeryd.1 | [2013-02-15 08:54:37,629: DEBUG/Beat] Celerybeat: Ticking with max interval->5.00 minutes
08:54:37 celeryd.1 | [2013-02-15 08:54:37,658: DEBUG/Beat] Celerybeat: Waking up in 5.00 minutes.
08:54:38 celeryd.1 | [2013-02-15 08:54:38,110: DEBUG/MainProcess] consumer: basic.qos: prefetch_count->16
08:54:38 celeryd.1 | [2013-02-15 08:54:38,126: DEBUG/MainProcess] consumer: Ready to accept tasks!
08:55:08 celeryd.1 | [2013-02-15 08:55:08,184: WARNING/MainProcess] Received and deleted unknown message. Wrong destination?!?
08:55:08 celeryd.1 | The full contents of the message body was: body: 26 (2b) {content_type:u'application/json' content_encoding:u'utf-8' delivery_info:{u'priority': 0, u'routing_key': u'celery', u'exchange': u'celery'}}

08:54:33 celeryd.1 |从pid 555开始
08:54:33 web.1 |从pid 554开始
08:54:33 web.1 |/usr/local/foreman/bin/foreman runner:第41行：exec:source:未找到
08:54:36网络1 | 2013-02-15 08:54:36[557][INFO]启动gunicorn 0.16.1
08:54:36网址| 2013-02-15 08:54:36[557][INFO]收听：http://0.0.0.0:5000 (557)
08:54:36 web.1 | 2013-02-15 08:54:36[557][INFO]使用worker:sync
08:54:36 web.1 | 2013-02-15 08:54:36[564][INFO]带pid的引导工作程序：564
08:54:36 web.1 | 2013-02-15 08:54:36[565][INFO]正在启动pid为的工作进程：565
08:54:36 web.1 | 2013-02-15 08:54:36[566][INFO]正在启动pid为的工作进程：566
08:54:36 web.1 | 2013-02-15 08:54:36[567][INFO]正在启动pid为的工作进程：567
正在加载模块。
celeryd.1 |[2013-02-15 08:54:37484:调试/主进程][Worker]声明组件。
celeryd.1 |[2013-02-15 08:54:37484:DEBUG/MainProcess][Worker]构建引导步骤图。
新的启动顺序：{ev、队列、节拍、池、中介、自动加载、计时器、状态数据库、自动缩放、消费者}
正在启动芹菜进程。。。
芹菜，芹菜，芹菜，芹菜！
08:54:37 celeryd.1 |[2013-02-15 08:54:37491:DEBUG/MainProcess]正在启动celery.concurrency.processs.TaskPool。。。
Celerybeat:开始。。。
08:54:37 celeryd.1 |[2013-02-15 08:54:37506:DEBUG/MainProcess]celery.concurrency.processs.TaskPool OK！
08:54:37 celeryd.1 |[2013-02-15 08:54:37507:调试/主进程]正在启动celery.worker.mediator.mediator。。。
celery.worker.mediator.mediator好的！
08:54:37 celeryd.1 |[2013-02-15 08:54:37507:调试/主进程]正在启动celery.worker.consumer.BlockingConsumer。。。
08:54:37芹菜1 |[2013-02-15 08:54:37508:警告/主进程]celery@myhost.local准备好的
消费者：重新建立与代理的连接。。。
celeryd.1 |[2013-02-15 08:54:37510:INFO/MainProcess]消费者：已连接到django://localhost//.
celeryd.1 |[2013-02-15 08:54:37628:调试/节拍]当前时间表：
08:54:37芹菜园1 | 5.00分钟
Celerybeat:5分钟后醒来。
08:54:38 celeryd.1 |[2013-02-15 08:54:38110:DEBUG/MainProcess]使用者：basic.qos:prefetch_count->16
消费者：准备好接受任务了！
08:55:08 celeryd.1 |[2013-02-15 08:55:08184:警告/主进程]收到并删除了未知消息。目的地错了？！？
08:55:08 celeryd.1 |消息正文的全部内容是：正文：26（2b）{content_类型：u'application/json'content_编码：u'utf-8'delivery_信息：{u'priority'：0，u'routing_key'：u'芹菜'，u'exchange'：u'芹菜'}

问题有两方面：

消息格式错误。根据@asksol提供的文档，它需要是一个字典，并遵循该页面底部的示例

示例消息

放置消息

Django==1.4.2
celery==3.0.15
django-celery==3.0.11
kombu==2.5.6

web: source bin/activate; python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 4; python manage.py syncdb
celeryd: python manage.py celeryd -E -B --loglevel=INFO

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  queue.put(id)
  queue.close()

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  message = {
    'task': 'process-next-task',
    'id': str(uuid.uuid4()),
    'args': [id],
    "kwargs": {},
    "retries": 0,
    "eta": str(datetime.datetime.now())
  }
  queue.put(message)
  queue.close()

Procfile进程是运行任务的使用者，因此不需要在任务中设置使用者。我只需要使用发布消息时发送的参数

api/tasks.py

# Celery configuration
import djcelery
CELERY_IMPORTS = ("api.tasks",)
BROKER_URL = "django://localhost//"
djcelery.setup_loader()

@task()
def process_next_task():
  with Connection(settings.BROKER_URL) as conn:
    queue = conn.SimpleQueue('celery')
    message = queue.get(block=True, timeout=1)
    id = int(message.payload)
    try:
      Model.objects.get(id=id)
    except Model.DoesNotExist:
      message.reject()
    else:
      # Do stuff here
      message.ack()
    queue.close()

@task(serializer='json', name='process-next-task')
def process_next_task(id):
  try:
    Model.objects.get(id=int(id))
  except Model.DoesNotExist:
    pass
  else:
    # Do stuff here

---

问题有两方面：

消息格式错误。根据@asksol提供的文档，它需要是一个字典，并遵循该页面底部的示例

示例消息

放置消息

Django==1.4.2
celery==3.0.15
django-celery==3.0.11
kombu==2.5.6

web: source bin/activate; python manage.py run_gunicorn -b 0.0.0.0:$PORT -w 4; python manage.py syncdb
celeryd: python manage.py celeryd -E -B --loglevel=INFO

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  queue.put(id)
  queue.close()

with Connection(settings.BROKER_URL) as conn:
  queue = conn.SimpleQueue('celery')
  message = {
    'task': 'process-next-task',
    'id': str(uuid.uuid4()),
    'args': [id],
    "kwargs": {},
    "retries": 0,
    "eta": str(datetime.datetime.now())
  }
  queue.put(message)
  queue.close()

Procfile进程是运行任务的使用者，因此不需要在任务中设置使用者。我只需要使用发布消息时发送的参数

api/tasks.py

# Celery configuration
import djcelery
CELERY_IMPORTS = ("api.tasks",)
BROKER_URL = "django://localhost//"
djcelery.setup_loader()

@task()
def process_next_task():
  with Connection(settings.BROKER_URL) as conn:
    queue = conn.SimpleQueue('celery')
    message = queue.get(block=True, timeout=1)
    id = int(message.payload)
    try:
      Model.objects.get(id=id)
    except Model.DoesNotExist:
      message.reject()
    else:
      # Do stuff here
      message.ack()
    queue.close()

@task(serializer='json', name='process-next-task')
def process_next_task(id):
  try:
    Model.objects.get(id=int(id))
  except Model.DoesNotExist:
    pass
  else:
    # Do stuff here

---

这不是这个问题的解决方案，
但标记在用celery4.0.2

输出如下：

[2017-02-09 17:45:12,136: WARNING/MainProcess] Received and deleted unknown message.  Wrong destination?!?

The full contents of the message body was: body: [[], {}, {u'errbacks': None, u'callbacks': None, u'chord': None, u'chain': None}] (77b)
{content_type:'application/json' content_encoding:'utf-8'
  delivery_info:{'consumer_tag': 'None4', 'redelivered': False, 'routing_key': 'test2', 'delivery_tag': 1L, 'exchange': ''} headers={'\xe5\xca.\xdb\x00\x00\x00\x00\x00': None, 'P&5\x07\x00': None, 'T\nKB\x00\x00\x00': '3f6295b3-c85c-4188-b424-d186da7e2edb', 'N\xfd\x17=\x00\x00': 'gen23043@hy-ts-bf-01', '\xcfb\xddR': 'py', '9*\xa8': None, '\xb7/b\x84\x00\x00\x00': 0, '\xe0\x0b\xfa\x89\x00\x00\x00': None, '\xdfR\xc4x\x00\x00\x00\x00\x00': [None, None], 'T3\x1d ': 'celeryserver.tasks.test', '\xae\xbf': '3f6295b3-c85c-4188-b424-d186da7e2edb', '\x11s\x1f\xd8\x00\x00\x00\x00': '()', 'UL\xa1\xfc\x00\x00\x00\x00\x00\x00': '{}'}}

解决方案：

---

谢谢你，这不是这个问题的解决方案，
但标记在用celery4.0.2

输出如下：

[2017-02-09 17:45:12,136: WARNING/MainProcess] Received and deleted unknown message.  Wrong destination?!?

The full contents of the message body was: body: [[], {}, {u'errbacks': None, u'callbacks': None, u'chord': None, u'chain': None}] (77b)
{content_type:'application/json' content_encoding:'utf-8'
  delivery_info:{'consumer_tag': 'None4', 'redelivered': False, 'routing_key': 'test2', 'delivery_tag': 1L, 'exchange': ''} headers={'\xe5\xca.\xdb\x00\x00\x00\x00\x00': None, 'P&5\x07\x00': None, 'T\nKB\x00\x00\x00': '3f6295b3-c85c-4188-b424-d186da7e2edb', 'N\xfd\x17=\x00\x00': 'gen23043@hy-ts-bf-01', '\xcfb\xddR': 'py', '9*\xa8': None, '\xb7/b\x84\x00\x00\x00': 0, '\xe0\x0b\xfa\x89\x00\x00\x00': None, '\xdfR\xc4x\x00\x00\x00\x00\x00': [None, None], 'T3\x1d ': 'celeryserver.tasks.test', '\xae\xbf': '3f6295b3-c85c-4188-b424-d186da7e2edb', '\x11s\x1f\xd8\x00\x00\x00\x00': '()', 'UL\xa1\xfc\x00\x00\x00\x00\x00\x00': '{}'}}

解决方案：

---

感谢

显然，librabbitmq问题与芹菜4.x中的新默认协议有关。如果您正在使用Django，您可以通过将

CELERY\u TASK\u protocol=1

放入设置中，或将

app.conf.TASK\u protocol=1

放入

celeryconf.py

然后你会