bash运算符为其他pythonoperator中使用的lib创建了一个新的错误

bash运算符为其他pythonoperator中使用的lib创建了一个新的错误,python,airflow,airflow-operator,Python,Airflow,Airflow Operator,在我的dag builder模块中,我有一组任务,该模块使用的Python操作符在Airflow中广泛使用。我正在kubernetes上使用docker部署气流 任务失败,并显示错误消息:没有名为pandas的模块。使用熊猫的其他任务也很成功 是的,我确实进入了容器(工人),发现pip3冻结确实出现了 2018-12-13 12:30:23,332] {bash_operator.py:87} INFO - Temporary script location: /tmp/airflowtmppk

在我的dag builder模块中,我有一组任务,该模块使用的Python操作符在Airflow中广泛使用。我正在kubernetes上使用docker部署气流

任务失败,并显示错误消息:
没有名为pandas的模块
。使用熊猫的其他任务也很成功

是的,我确实进入了容器(工人),发现
pip3冻结
确实出现了

2018-12-13 12:30:23,332] {bash_operator.py:87} INFO - Temporary script location: /tmp/airflowtmppkovwfth/pscript_pclean_zjg4qfamp9pda9jsxysyrqfj_AWFtK5ucowyw2
[2018-12-13 12:30:23,333] {bash_operator.py:97} INFO - Running command: python /usr/local/airflow/rootfs/mopng_baseline_v2/scripts/pclean_zjg4qfamp9pda9jsxysyrqfj_AWFtK.py /usr/local/airflow/rootfs/mopng_baseline_v2/scheduled__2018-12-12T14:00:00+00:00/appended/DsDnV0TjSHnL0DF53JLjmUtO.csv /usr/local/airflow/rootfs/mopng_baseline_v2/scheduled__2018-12-12T14:00:00+00:00/pcleaned/ztYVV9nkh5t425gYjFqKuAD9.csv
[2018-12-13 12:30:23,344] {bash_operator.py:106} INFO - Output:
[2018-12-13 12:30:23,359] {bash_operator.py:110} INFO - Traceback (most recent call last):
[2018-12-13 12:30:23,359] {bash_operator.py:110} INFO -   File "/usr/local/airflow/rootfs/mopng_baseline_v2/scripts/pclean_zjg4qfamp9pda9jsxysyrqfj_AWFtK.py", line 3, in <module>
[2018-12-13 12:30:23,359] {bash_operator.py:110} INFO -     import pandas as pd
[2018-12-13 12:30:23,360] {bash_operator.py:110} INFO - ImportError: No module named pandas
[2018-12-13 12:30:23,362] {bash_operator.py:114} INFO - Command exited with return code 1
[2018-12-13 12:30:23,383] {models.py:1736} ERROR - Bash command failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/airflow/models.py", line 1633, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.5/dist-packages/airflow/operators/bash_operator.py", line 118, in execute
    raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
2018-12-13 12:30:23332]{bash_operator.py:87}信息-临时脚本位置:/tmp/airflowtmppkowfth/pscript_pclean\u zjg4qfamp9pda9jsxysyrqfj_awftk5ucow2
[2018-12-13 12:30:23333]{bash_operator.py:97}信息-运行命令:python/usr/local/aiffair/rootfs/mopng_baseline_v2/scripts/pclean_zjg4qfamp9pda9jsxysyrqfj_AWFtK.py/usr/local/aiffair/rootfs/mopng_baseline_v2/scheduled_2018-12-12T14:00+00:00/added/DsDnV0TjSHnL0DF53JLjmUtO.csv/usr/local/airflow/rootfs/mopng_baseline_v2/scheduled__2018-12-12T14:00:00+00:00/pclean/ztyv9nkh5t425gyjfqkuad9.csv
[2018-12-13 12:30:23344]{bash_operator.py:106}信息-输出:
[2018-12-13 12:30:23359]{bash_operator.py:110}信息-回溯(最近一次呼叫最后一次):
[2018-12-13 12:30:23359]{bash_operator.py:110}INFO-File“/usr/local/aiffort/rootfs/mopng_baseline_v2/scripts/pclean_zjg4qfamp9pda9jsxysyrqfj_AWFtK.py”,第3行,in
[2018-12-13 12:30:23359]{bash_operator.py:110}信息-作为pd导入熊猫
[2018-12-13 12:30:23360]{bash_operator.py:110}INFO-ImportError:没有名为pandas的模块
[2018-12-13 12:30:23362]{bash_operator.py:114}INFO-命令已退出,返回代码为1
[2018-12-13 12:30:23383]{models.py:1736}错误-Bash命令失败
回溯(最近一次呼叫最后一次):
文件“/usr/local/lib/python3.5/dist-packages/afflow/models.py”,第1633行,在原始任务中
结果=任务\复制.执行(上下文=上下文)
文件“/usr/local/lib/python3.5/dist-packages/afflow/operators/bash_operator.py”,第118行,在execute中
raise AirflowException(“Bash命令失败”)
airflow.Exception.AirflowException:Bash命令失败

失败的操作符不是
PythonOperator
,而是
bash操作符。最可能的原因是Bash中的
python
当前指向的是一个不同于当前运行的python环境


请确保在
bash操作符中指定
python3
,或者在与
PythonOperator
相同的环境中从命令行调用Python所需的任何额外配置。

我将cmd命令更改为在执行此命令的Python文件中使用
python3
。但是,当我使用docker将代码推送到kubernetes集群时,没有反映出任何更改。我在构建docker映像时使用了无缓存。我也应该杀死豆荚吗?我从来没有在K8上使用过气流,但我想是的。如果有疑问,请重新创建集群以使用空白板进行测试。