Python 芹菜任务将结果返回到父任务
我有一个芹菜网抓取任务,我想把所有创建的子对象返回给父对象。我怎样才能把这些ID还给家长 我正在尝试做的粗略示例Python 芹菜任务将结果返回到父任务,python,celery,Python,Celery,我有一个芹菜网抓取任务,我想把所有创建的子对象返回给父对象。我怎样才能把这些ID还给家长 我正在尝试做的粗略示例 @task def scrape_page(url): page = Page.objects.create( name='new_page' ) response = requests.get(url) children = parse_children(response) for child in children:
@task
def scrape_page(url):
page = Page.objects.create(
name='new_page'
)
response = requests.get(url)
children = parse_children(response)
for child in children:
result = scrape_child.delay(child.url)
page.childen.add(result)
@task
def scrape_child(url):
response = requests.get(url)
parsed = parse_child(response)
child = Child.objects.create(**parsed)
return child
您不应该在任务中等待延迟的任务,您可以使用
.apply()
同步调用该任务,但这不会调用单独的任务。
编辑:我找到了文档,请参阅
也许你应该用。一种方法是延迟组scrape\u child
任务并发送父级page\u id
from celery import group
@task
def scrape_page(url):
page = Page.objects.create(
name='new_page'
)
response = requests.get(url)
children = parse_children(response)
children_group = group([
scrape_child.s(page.id, child.url) # using the signature
for child in children
])
children_group.delay()
return True
@task
def scrape_child(page_id, url):
response = requests.get(url)
parsed = parse_child(response)
child = Child.objects.create(**parsed)
child.page_id = page_id
return child