Airflow 气流-在同一DAG中使用任务组和Python操作符
我目前正在使用Airflow Taskflow API 2.0。我遇到了一个问题,即如何结合使用TaskGroup和BranchPythonOperator 下面是我的代码:Airflow 气流-在同一DAG中使用任务组和Python操作符,airflow,airflow-scheduler,Airflow,Airflow Scheduler,我目前正在使用Airflow Taskflow API 2.0。我遇到了一个问题,即如何结合使用TaskGroup和BranchPythonOperator 下面是我的代码: import airflow from airflow.models import DAG from airflow.decorators import task, dag from airflow.operators.dummy_operator import DummyOperator from airflow.ope
import airflow
from airflow.models import DAG
from airflow.decorators import task, dag
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import BranchPythonOperator, PythonOperator
from airflow.operators.python import task, get_current_context
from random import randint
from airflow.utils.task_group import TaskGroup
default_args = {
'owner': 'Airflow',
'start_date': airflow.utils.dates.days_ago(2),
}
@task
def dummy_task():
return {}
@task
def task_b():
return {}
@task
def task_c():
return {}
def final_step():
return {}
def get_tasks(**kwargs):
task = 'task_a'
return task
with DAG(dag_id='branch_dag',
default_args=default_args,
schedule_interval=None) as dag:
with TaskGroup('task_a') as task_a:
obj = dummy_task()
tasks = BranchPythonOperator(
task_id='check_api',
python_callable=get_tasks,
provide_context=True
)
final_step = PythonOperator(
task_id='final_step',
python_callable=final_step,
trigger_rule='one_success'
)
b = task_b()
c = task_c()
tasks >> task_a >> final_step
tasks >> b >> final_step
tasks >> c >> final_step
当我触发此DAG时,我在check_api任务中得到以下错误:
aiffort.exceptions.TaskNotFound:未找到任务
是否可以将TaskGroup与BranchPythonOperator结合使用以使其正常工作
谢谢,BranchPythonOperator将返回任务ID 您需要将
get_tasks
功能更改为:
def get_tasks(**kwargs):
task = 'task_a.dummy_task'
return task
太棒了,这真有效!非常感谢埃拉德:)