Airflow 尝试通过SSHHook执行远程脚本时出错

使用airflow，我试图通过SSHHook执行远程脚本。脚本就是这样

echo "this is a test"

在远程机器内部，我可以通过“bash测试”来运行它

我创建了一个气流脚本，如下所示：

import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
from airflow.contrib.operators.ssh_operator import SSHOperator
from airflow.contrib.hooks.ssh_hook import SSHHook

# add a new SSH connection using the WEB UI under the admin --> connections tab.
sshHook = SSHHook(ssh_conn_id="test_ssh")

# Following are defaults which can be overridden later on
default_args = {
    'owner': 'tester',
    'depends_on_past': False,
    'start_date': datetime(2019,6,24),
    'email': ['user123@gmail.com'],
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 1,
    'retry_delay': timedelta(minutes=1),
}

dag = DAG('test', default_args=default_args)

t1 = SSHOperator(
    ssh_conn_id= sshHook,
    task_id='test_20190624',
    command='bash /home/tester/run_test',
    dag=dag)

[2019-06-24 11:27:17,790] {ssh_operator.py:80} INFO - ssh_hook is not provided or invalid. Trying ssh_conn_id to create SSHHook.
[2019-06-24 11:27:17,792] {__init__.py:1580} ERROR - SSH operator error: 'SSHHook' object has no attribute 'upper'
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/operators/ssh_operator.py", line 82, in execute
    timeout=self.timeout)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/hooks/ssh_hook.py", line 90, in __init__
    conn = self.get_connection(self.ssh_conn_id)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 80, in get_connection
    conn = random.choice(cls.get_connections(conn_id))
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 71, in get_connections
    conn = cls._get_connection_from_env(conn_id)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 63, in _get_connection_from_env
    environment_uri = os.environ.get(CONN_ENV_PREFIX + conn_id.upper())
AttributeError: 'SSHHook' object has no attribute 'upper'

然后我得到了这样一个错误：

import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
from airflow.contrib.operators.ssh_operator import SSHOperator
from airflow.contrib.hooks.ssh_hook import SSHHook

# add a new SSH connection using the WEB UI under the admin --> connections tab.
sshHook = SSHHook(ssh_conn_id="test_ssh")

# Following are defaults which can be overridden later on
default_args = {
    'owner': 'tester',
    'depends_on_past': False,
    'start_date': datetime(2019,6,24),
    'email': ['user123@gmail.com'],
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 1,
    'retry_delay': timedelta(minutes=1),
}

dag = DAG('test', default_args=default_args)

t1 = SSHOperator(
    ssh_conn_id= sshHook,
    task_id='test_20190624',
    command='bash /home/tester/run_test',
    dag=dag)

[2019-06-24 11:27:17,790] {ssh_operator.py:80} INFO - ssh_hook is not provided or invalid. Trying ssh_conn_id to create SSHHook.
[2019-06-24 11:27:17,792] {__init__.py:1580} ERROR - SSH operator error: 'SSHHook' object has no attribute 'upper'
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/operators/ssh_operator.py", line 82, in execute
    timeout=self.timeout)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/contrib/hooks/ssh_hook.py", line 90, in __init__
    conn = self.get_connection(self.ssh_conn_id)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 80, in get_connection
    conn = random.choice(cls.get_connections(conn_id))
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 71, in get_connections
    conn = cls._get_connection_from_env(conn_id)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 63, in _get_connection_from_env
    environment_uri = os.environ.get(CONN_ENV_PREFIX + conn_id.upper())
AttributeError: 'SSHHook' object has no attribute 'upper'

---

您应该直接使用SSH连接ID，或者只使用SSHHook。这里的问题是你把两者都混在一起了

1） 使用SSHHook：

t1 = SSHOperator(
    ssh_hook = sshHook,
    task_id='test_20190624',
    command='bash /home/tester/run_test',
    dag=dag)

2） 直接使用SSH连接：

t1 = SSHOperator(
    ssh_conn_id="test_ssh",
    task_id='test_20190624',
    command='bash /home/tester/run_test',
    dag=dag)