Amazon web services AWS上的Sqoop作业失败，连接错误_Amazon Web Services_Sqoop

Amazon web services AWS上的Sqoop作业失败，连接错误

amazon-web-services

Amazon web services AWS上的Sqoop作业失败，连接错误,amazon-web-services,sqoop,Amazon Web Services,Sqoop,我正在执行下面的sqoop命令，以从另一个aws rds实例获取一个表，并将其转移到hdfs #!/bin/bash sqoop import \ --connect jdbc:mysql://awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com/financials_data \ --username someuser \ --password somepwd \ --table member_score \ --m 1 \ --target-dir

我正在执行下面的sqoop命令，以从另一个aws rds实例获取一个表，并将其转移到hdfs

#!/bin/bash

sqoop import \
--connect jdbc:mysql://awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com/financials_data \
--username someuser \
--password somepwd \
--table member_score \
--m 1 \
--target-dir /capstone/member_score

我可以使用工作台连接到此服务器。但是，sqoop无法获取数据

堆栈跟踪如下所示：

[ec2-user@ip-10-0-0-238 capstone]$ ./DataIngestion.txt
Warning: /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/01/03 03:56:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.15.1
20/01/03 03:56:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/01/03 03:56:45 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/01/03 03:56:45 INFO tool.CodeGenTool: Beginning code generation
20/01/03 03:58:52 ERROR manager.SqlManager: Error executing statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at

堆栈跟踪显示连接错误。但是，我可以使用Mysql Workbench进行连接，因为sqoop给出了一个连接错误，所以我尝试ping服务器

[ec2-user@ip-10-0-0-238 capstone]$ ping awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com
PING ec2-3-211-175-82.compute-1.amazonaws.com (3.211.175.82) 56(84) bytes of data.
^C
--- ec2-3-211-175-82.compute-1.amazonaws.com ping statistics ---
2935 packets transmitted, 0 received, 100% packet loss, time 2934021ms

看到无法访问服务器时，下一步要检查的是AWS上的

安全组设置
出站规则应允许所有流量


我以前将出站设置为特定IP。由于每次静态IP不起作用时AWS都会分配新的IP。
由于sqoop发出连接错误，我尝试ping服务器
[ec2-user@ip-10-0-0-238 capstone]$ ping awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com
PING ec2-3-211-175-82.compute-1.amazonaws.com (3.211.175.82) 56(84) bytes of data.
^C
--- ec2-3-211-175-82.compute-1.amazonaws.com ping statistics ---
2935 packets transmitted, 0 received, 100% packet loss, time 2934021ms

看到无法访问服务器时，下一步要检查的是AWS上的安全组设置
出站规则应允许所有流量


我以前将出站设置为特定IP。因为每次静态IP不起作用时AWS都会分配新的IP。
你能告诉我你的hadoop集群在哪里吗？在prem还是EMR上？如果是，您是否尝试过从集群远程登录rds，以及是否可以验证允许来自hadoop集群的流量的安全组配置？谢谢您的回复@bdcloud。hdfs也在AWS EC2上。我正在AWS上运行Cloudera虚拟机。我试图通过ssh连接aws机器，但连接超时。但是，同事们能够接触到这台机器。我已经查看了安全组设置，端口（3306）应该可以访问。AWS RDS似乎无法访问。见此：[ec2-user@ip-10-0-0-238 capstone]$mysql-h upgradawsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com-u upgraduser-p输入密码：错误2003（HY000）：无法连接到“upgradawsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com”（110）上的mysql服务器[ec2]-user@ip-10-0-0-238顶石]$AWS可以从特定ec2实例选择性地不可访问吗？您能告诉我您的hadoop集群在哪里吗？在prem还是EMR上？如果是，您是否尝试过从集群远程登录rds，以及是否可以验证允许来自hadoop集群的流量的安全组配置？谢谢您的回复@bdcloud。hdfs也在AWS EC2上。我正在AWS上运行Cloudera虚拟机。我试图通过ssh连接aws机器，但连接超时。但是，同事们能够接触到这台机器。我已经查看了安全组设置，端口（3306）应该可以访问。AWS RDS似乎无法访问。见此：[ec2-user@ip-10-0-0-238 capstone]$mysql-h upgradawsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com-u upgraduser-p输入密码：错误2003（HY000）：无法连接到“upgradawsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com”（110）上的mysql服务器[ec2]-user@ip-10-0-0-238顶石]$AWS是否可以从特定ec2实例选择性地不可访问？