Airflow 没有这样的文件或目录/afflow/xcom/return.json
创建了一个包含Airflow 没有这样的文件或目录/afflow/xcom/return.json,airflow,apache-airflow-xcom,Airflow,Apache Airflow Xcom,创建了一个包含/afflow/xcom/return.json 所有子目录上的chmod+x 由于日志显示它找不到文件或目录(chmod+x) 这是日志 [2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - [2019-03-18 20:32:07,007] {pod_launcher.py:166} INFO - Running command... cat /airflow/xcom/return.json [2019-03-18
/afflow/xcom/return.json
所有子目录上的chmod+x
由于日志显示它找不到文件或目录(chmod+x)
这是日志
[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO - [2019-03-18
20:32:07,007] {pod_launcher.py:166} INFO - Running command... cat
/airflow/xcom/return.json
[2019-03-18 20:32:07,007] {logging_mixin.py:95} INFO -
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18
20:32:07,026] {pod_launcher.py:173} INFO - cat: can't open
'/airflow/xcom/return.json': No such file or directory
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO -
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO - [2019-03-18
20:32:07,026] {pod_launcher.py:166} INFO - Running command... kill -s
SIGINT 1
[2019-03-18 20:32:07,026] {logging_mixin.py:95} INFO -
[2019-03-18 20:32:07,067] {models.py:1788} ERROR - Pod Launching
failed: Failed to extract xcom from pod: fail-e18e3dac
所以尝试了这种方式,它可以工作,但这意味着它从外部分配xcom json,而不是从映像分配
return_value = '{"foo": "bar"\n, "buzz": 2}'
strtpodbefore = KubernetesPodOperator(namespace='rguonew',
image="python:3.6.6-stretch",
cmds=["bash", "-cx"],
name="fail",
task_id="failing-task",
arguments=['echo \'{}\' >
/airflow/xcom/return.json'.format(return_value)],
get_logs= True,
xcom_push=True,
dag=dag
)
所以我尝试了最后的解决方案,添加了一个额外的参数,但仍然不起作用,第一个命令没有得到这样的目录返回
strtpodbefore = KubernetesPodOperator(namespace='rguonew',
image="myimages",
name="fail",
image_pull_policy="Always",
cmds=["bash", "-cx"],
arguments=['echo \'{}\' >
/airflow/xcom/return.json'.format(return_value)],
task_id="failing-task",
get_logs= True,
xcom_push=True,
dag=dag
)
当您将xcom_push设置为True时,将与executor容器一起创建一个sidecar容器。sidecar容器将读取执行器的
/afflow/xcom/return.json
,因此您实际上需要像在第二个示例中所做的那样,从执行器容器中写入它
请看这里:
当我们在pod xcom上工作时,您还需要设置一些与RBAC相关的问题:
所以我想sidecar不仅仅是像我在第二个示例中那样注入数据,而是用第一个pod控制所有的文件操作符是的,你在第一个pod中写入json文件,sidecar抓取写入的文件并将其推送到xcom。
strtpodbefore = KubernetesPodOperator(namespace='rguonew',
image="myimages",
name="fail",
image_pull_policy="Always",
cmds=["bash", "-cx"],
arguments=['echo \'{}\' >
/airflow/xcom/return.json'.format(return_value)],
task_id="failing-task",
get_logs= True,
xcom_push=True,
dag=dag
)