如何在Airflow操作符中的Airflow XCom_pull值上应用Python函数
我有一个需求,需要通过SQLsensor操作符捕获XCOM响应,并应用一些python命令来更改数据格式 我拥有的代码如下所示:-如何在Airflow操作符中的Airflow XCom_pull值上应用Python函数,python,airflow,airflow-operator,apache-airflow-xcom,Python,Airflow,Airflow Operator,Apache Airflow Xcom,我有一个需求,需要通过SQLsensor操作符捕获XCOM响应,并应用一些python命令来更改数据格式 我拥有的代码如下所示:- delta_sql_sensor_task = SqlSensor( task_id= "Verify-Completion-of-Delta-Job", poke_interval= 10, timeout=100, conn_id="lct_statedb", #gobal_i
delta_sql_sensor_task = SqlSensor(
task_id= "Verify-Completion-of-Delta-Job",
poke_interval= 10,
timeout=100,
conn_id="lct_statedb",
#gobal_id_list="{{ str(ti.xcom_pull(key='id_list', task_ids=['hook_task'])).replace('[','') }}",
sql="SELECT COALESCE ( (SELECT count(*) FROM lct_transformation_history WHERE deltaload_transformation_status is not null AND fullload_transformation_status is not null AND category='sites' AND id in (" + "{{ (ti.xcom_pull(key='id_list', task_ids=['hook_task'])) }}" + ") group by category),0) AS TRIGGER;",
trigger_rule='one_failed',
dag=dag)
我从代码{{ti.xcom\u pullkey='id\u list',task\u id=['hook\u task'].str}}收到的响应为['13',],但我需要将响应更改为'13'
以下是我尝试过的选项:-
delta_sql_sensor_task = SqlSensor(
task_id= "Verify-Completion-of-Delta-Job",
poke_interval= 10,
timeout=100,
conn_id="lct_statedb",
#gobal_id_list="{{ str(ti.xcom_pull(key='id_list', task_ids=['hook_task'])).replace('[','') }}",
sql="SELECT COALESCE ( (SELECT count(*) FROM lct_transformation_history WHERE deltaload_transformation_status is not null AND fullload_transformation_status is not null AND category='sites' AND id in (" + "{{ (ti.xcom_pull(key='id_list', task_ids=['hook_task'])) }}" + ") group by category),0) AS TRIGGER;",
trigger_rule='one_failed',
dag=dag)
我尝试将xcom_pull响应存储在全局变量gobal_id_列表中,并应用Python函数,但没有成功。
我还尝试在SQL变量中应用Python运算符,如下所示,但运气不佳,代码如下所示:-
sql=SELECT COALESCE SELECT count*FROM lct_transformation_历史记录,其中deltaload_transformation_状态不为null,fullload_transformation_状态不为null,category='sites'和id位于+{{strti.xcom_pullkey='id_list',task_id=['hook_task']。替换'[',.replace'],.replace,.replace,.replace',}}+按类别分组,0作为触发器;,。
期待有价值的投入。
谢谢。我已经用以下解决方案解决了我的问题,因为我的Xcom响应是在tuple&list中,因此我通过下面给出的代码更改解决了问题:-
"{{ (ti.xcom_pull(key='id_list', task_ids=['hook_task'])).str()[0][0][0] }}
它给了我期望的结果,我期望的是13