在新项目中重用Django芹菜应用程序,找不到其他项目模块或芹菜应用程序

在新项目中重用Django芹菜应用程序,找不到其他项目模块或芹菜应用程序,django,celery,django-celery,Django,Celery,Django Celery,我有一个工作应用程序,它使用芹菜来执行后台文件传输和其他数据收集任务(glint),我想在我的新项目中使用这个应用程序的一部分。我尝试了几乎所有我能想到的配置排列,但问题仍然存在。以下是运行时的错误转储: 服务芹菜开始 Traceback (most recent call last): File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) Fil

我有一个工作应用程序,它使用芹菜来执行后台文件传输和其他数据收集任务(glint),我想在我的新项目中使用这个应用程序的一部分。我尝试了几乎所有我能想到的配置排列,但问题仍然存在。以下是运行时的错误转储:

服务芹菜开始

 Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 54, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 793, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 309, in execute_from_commandline
    argv = self.setup_app_from_commandline(argv)
  File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
    self.app = self.find_app(app)
  File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 489, in find_app
    return find_app(app, symbol_by_name=self.symbol_by_name)
  File "/usr/lib/python3.6/site-packages/celery/app/utils.py", line 235, in find_app
    sym = symbol_by_name(app, imp=imp)
  File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 492, in symbol_by_name
    return symbol_by_name(name, imp=imp)
  File "/usr/lib/python3.6/site-packages/kombu/utils/__init__.py", line 96, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/usr/lib/python3.6/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
    return imp(module, package=package)
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/cloudscheduler/web_frontend/cloudscheduler/glintwebui/celery_app.py", line 26, in <module>
    django.setup()
  File "/usr/lib64/python3.6/site-packages/django/__init__.py", line 22, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/usr/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/usr/lib64/python3.6/site-packages/django/conf/__init__.py", line 41, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/lib64/python3.6/site-packages/django/conf/__init__.py", line 110, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'cloudscheduler_web'
    > w4@csv2.heprc.uvic.ca: * Child terminated with errorcode 1
FAILED
环境文件:

# Name of nodes to start
# here we have a single node
#CELERYD_NODES="w1"
# or we could have three nodes:
CELERYD_NODES="w1 w2 w3 w4"

# We only want each worker to claim 1 job at a time, especially the worker dedicated to img collection
CELERYD_PREFETCH_MULTIPLIER=1

# Absolute path to "manage.py"
CELERY_BIN="/opt/cloudscheduler/web_frontend/cloudscheduler/manage.py"

# Where to chdir at start. This could be the root of a virtualenv.
CELERYD_CHDIR="/opt/cloudscheduler/web_frontend/cloudscheduler/glintwebui"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="celery_app"
# or fully qualified:
#CELERY_APP="glintwebui:celery_app"


# How to call manage.py
CELERYD_MULTI="celery multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="-Q:w1 image_collection -Q:w2,w3,w4 celery -A celery_app --concurrency=1 -Ofair"

# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"
这里需要注意的是,当我将CELERYD_CHDIR更改为顶级项目目录时,它找不到应用程序。当chdir被设置为顶级项目时,我尝试了几个不同的表达式来表示完全合格的芹菜应用程序,但它永远也找不到芹菜应用程序

以下是cloudscheduler_web目录中main settings.pt的相关设置:

CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Canada/Pacific'
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_DEFAULT_QUEUE = 'celery'
CELERY_DEFAULT_EXCHANGE = "celery"
CELERY_QUEUES = {
    "celery": {"exchange": "celery"},
    "image_collection": {"exchange": "image_collection"},
}
CELERY_ROUTES = {
    'cloudscheduler.glintwebui.tasks.image_collection': {'queue': 'image_collection'},
}
最后是项目的目录结构:

cloudscheduler
│
├── cloudscheduler_web
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── csv2
│   ├── apps.py
│   ├── cloud_views.py
│   ├── config.py
│   ├── config.pyc
│   ├── csv2_web.yaml
│   ├── group_views.py
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   ├── user_views.py
│   ├── views.py
│   └── view_utils.py
├── glintwebui
│   ├── admin.py
│   ├── apps.py
│   ├── celery_app.py
│   ├── config.py
│   ├── forms.py
│   ├── glint_api.py
│   ├── __init__.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   ├── utils.py
│   ├── __version__.py
│   └── views.py
└── manage.py
我迷路的地方是芹菜如何找到芹菜应用程序或其他应用程序模块,但决不能同时找到两者。我已经阅读了芹菜配置文档好几次了,我觉得我还是缺少了一些东西,这才是它的表现方式

如果有帮助的话,我可以发布原始应用程序(glint)的目录结构。作为奖励,这里是芹菜app.py文件的一部分:

from __future__ import absolute_import, unicode_literals
import os
import time
import subprocess
import django
from django.conf import settings

from celery import Celery
from celery.utils.log import get_task_logger
#import glintwebui.config as config
import config

from glint_api import repo_connector

logger = get_task_logger(__name__)

# Indicate Celery to use the default Django settings module
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'cloudscheduler_web.settings')

django.setup()

app = Celery('celery_app', broker=config.celery_url, backend=config.celery_backend)
app.config_from_object('django.conf:settings')


@app.task(bind=True)
def debug_task(self):
    logger.debug('Request: {0!r}'.format(self.request))


@app.task(bind=True)
def image_collection(self):
尝试py-D建议后出错:

Traceback (most recent call last):
      File "/usr/lib/python3.6/site-packages/celery/app/utils.py", line 241, in find_app
        found = sym.app
    AttributeError: module 'glintwebui' has no attribute 'app'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 54, in <module>
        main()
      File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 30, in main
        main()
      File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 81, in main
        cmd.execute_from_commandline(argv)
      File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 793, in execute_from_commandline
        super(CeleryCommand, self).execute_from_commandline(argv)))
      File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 309, in execute_from_commandline
        argv = self.setup_app_from_commandline(argv)
      File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
        self.app = self.find_app(app)
      File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 489, in find_app
        return find_app(app, symbol_by_name=self.symbol_by_name)
      File "/usr/lib/python3.6/site-packages/celery/app/utils.py", line 246, in find_app
        found = sym.celery
    AttributeError: module 'glintwebui' has no attribute 'celery'
        > w4@csv2.heprc.uvic.ca: * Child terminated with errorcode 1
    FAILED
回溯(最近一次呼叫最后一次):
文件“/usr/lib/python3.6/site packages/芹菜/app/utils.py”,第241行,在find_应用程序中
found=sym.app
AttributeError:模块“glintwebui”没有属性“app”
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/usr/lib64/python3.6/runpy.py”,第193行,在“运行”模块中作为“主”
“\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu
文件“/usr/lib64/python3.6/runpy.py”,第85行,在运行代码中
exec(代码、运行\全局)
文件“/usr/lib/python3.6/site packages/cellery/_main__.py”,第54行,在
main()
文件“/usr/lib/python3.6/site packages/cellery/_main__.py”,第30行,在main中
main()
文件“/usr/lib/python3.6/site packages/celery/bin/celery.py”,第81行,在main中
cmd.从命令行(argv)执行命令
文件“/usr/lib/python3.6/site packages/celery/bin/celery.py”,第793行,从命令行执行
超级(CeleryCommand,self)。从命令行(argv)执行命令)
文件“/usr/lib/python3.6/site packages/celery/bin/base.py”,第309行,在命令行中执行
argv=self.setup\u app\u from\u命令行(argv)
文件“/usr/lib/python3.6/site packages/芹菜/bin/base.py”,第469行,在安装程序中,从命令行开始
self.app=self.find_应用程序(app)
文件“/usr/lib/python3.6/site packages/芹菜/bin/base.py”,第489行,在find_应用程序中
返回find\u app(app,symbol\u by\u name=self.symbol\u by\u name)
文件“/usr/lib/python3.6/site packages/芹菜/app/utils.py”,第246行,在find_应用程序中
找到的=西芹
AttributeError:模块“glintwebui”没有属性“芹菜”
> w4@csv2.heprc.uvic.ca:*以错误代码1终止的子项
失败

你必须选择
应用程序
标题
glintwebui
,而不是
芹菜应用程序
。 请尝试使用:

app = Celery('glintwebui', broker=config.celery_url, backend=config.celery_backend)

在尝试了一百万次之后,我终于发现我的init.py文件缺少芹菜应用程序导入。我想只要你的代码与python2-3兼容,你就可以在新的python3项目中重用它。

@Alasdair正如我在问题中提到的那样,我已经尝试过了。一旦我切换到顶级目录芹菜似乎失去了芹菜应用程序的轨道。我的想法是可能存在“芹菜应用程序”的问题,但我尝试了几乎所有我能想到的排列,结果是不变的:芹菜应用程序=“芹菜应用程序”芹菜应用程序=“glintwebui.CELERY\u应用程序:应用程序”芹菜应用程序=“glintwebui.CELERY\u应用程序:芹菜应用程序”芹菜应用程序=“glintwebui:CELERY\u应用程序”我已经更新了这个问题,使错误转储在问题的末尾。Hrm,它似乎找到了glintwebui模块,但仍然没有找到应用定义所在的芹菜app.py文件。我已经在问题的底部添加了新的错误。我不确定我需要做什么才能使find_应用程序在芹菜_app.py文件中显示。请查看它是否对您有用。。看了那里的所有东西,但都没用。在做了大量的挖掘之后,我认为整个事情都是版本不匹配的问题。我使用Python2开发了最初的应用程序,而新的应用程序使用的是python3,所以我想我可能需要从头开始。(也从芹菜4.0.2增加到芹菜4.10)
app = Celery('glintwebui', broker=config.celery_url, backend=config.celery_backend)