Airflow 如何在dag中将pythonoperator任务的参数传递给SimpleHttOperator任务?

Airflow 如何在dag中将pythonoperator任务的参数传递给SimpleHttOperator任务?,airflow,apache-airflow,airflow-scheduler,Airflow,Apache Airflow,Airflow Scheduler,我想触发一个simplehttpoperator,如下所示: 气流触发器测试触发器--conf'{“name”:“something”} 然后我使用pythonoperator python_callable通过使用kwargs['dag_run'].conf接受参数,我想将['dag_run'].conf传递给simplehttpoperator,我该如何做?有人能帮忙吗 cc_ = {} def run_this_func(ds, **kwargs): cc_ = kwargs[

我想触发一个simplehttpoperator,如下所示: 气流触发器测试触发器--conf'{“name”:“something”}

然后我使用pythonoperator python_callable通过使用kwargs['dag_run'].conf接受参数,我想将['dag_run'].conf传递给simplehttpoperator,我该如何做?有人能帮忙吗

cc_ = {}


def run_this_func(ds, **kwargs):
    cc_ = kwargs['dag_run'].conf
    logging.info(cc_)
    return cc_

run_this = PythonOperator(
    task_id='run_this',
    provide_context=True,
    python_callable=run_this_func,
    dag=dag)

http_task = SimpleHttpOperator(
    task_id='http_task',
    http_conn_id='test_http',
    method='POST',
    endpoint='/api/v1/function',
    data=cc_,
    headers={"Authorization": "Basic YWRtaW46MTIzNDU2", "Accept": "application/json, text/plain, */*"},
    response_check=lambda response: True if "10000" in response.content else False,
    dag=dag)

http_task.set_upstream(run_this)

对于任务之间的通信,您可能需要检查XCOM

*****更新******
(感谢Daniel提供更多详细信息) 下面是一些代码,您可以尝试一下,在SimpleHttpOperator中,您可以通过XCOM获得返回值:

http_task = SimpleHttpOperator(
    task_id='http_task',
    http_conn_id='test_http',
    method='POST',
    endpoint='/api/v1/function',
    data=json.loads("{{ task_instance.xcom_pull(task_ids='run_this', key='return_value') }}"),
    headers={"Authorization": "Basic YWRtaW46MTIzNDU2", "Accept": "application/json, text/plain, */*"},
    response_check=lambda response: True if "10000" in response.content else False,
    dag=dag)

感谢@Chengzhi和@Daniel。 最后,我在Jinja2/filter.py中编写了一个自定义过滤器“tojson”,因为默认的Jinja2版本是2.8.1,而Jinja2直到2.9版本才包含名为“tojson”的内置过滤器

def do_tojson(value):
    value = json.JSONEncoder().encode(value)
    return value
在dag文件中,代码如下所示。它起作用了

def run_this_func(ds, **kwargs):
    cc_ = kwargs['dag_run'].conf
    return cc_

run_this = PythonOperator(
    task_id='run_this',
    provide_context=True,
    python_callable=run_this_func,
    dag=dag)

http_task = SimpleHttpOperator(
    task_id='http_task',
    http_conn_id='test_http',
    method='POST',
    endpoint='/api/v1/task',
    data="{{ task_instance.xcom_pull(task_ids='run_this') |tojson}}",
    headers={"Authorization": "Basic YWRtaW46MTIzNDU2", "Accept": "application/json, text/plain, */*",
             "Content-Type": "application/json"},
    response_check=lambda response: True if "10000" in response.content else False,
    dag=dag)

http_task.set_upstream(run_this)

但是如何在simplehttpoperator中使用XCOM呢?你能给出任何案例代码吗?@pyfroggogogo,我更新了一些代码示例,如果必须将此工作模板作为字符串传递,请尝试。可以使用
data=json.loads({{…| tojson}}}')
在呈现后将其返回到dict类型。