Java Hadoop3:连接到ResourceManager的工作节点错误

Java Hadoop3:连接到ResourceManager的工作节点错误,java,hadoop,mapreduce,yarn,hadoop-streaming,Java,Hadoop,Mapreduce,Yarn,Hadoop Streaming,我有一个3节点的hadoop集群(DigitalOcean): hadoop主机配置为namenode和datanode hadoop-worker1和hadoop-worker2配置为数据节点 每当我运行mapreduce流作业并且选择工作节点来运行ApplicationMaster时,该作业将在尝试连接到ResourceManager时挂起。datanode日志显示它试图连接到0.0.0.0 INFO [main] org.apache.hadoop.yarn.client.RMProx

我有一个3节点的hadoop集群(DigitalOcean):

  • hadoop主机配置为namenode和datanode
  • hadoop-worker1和hadoop-worker2配置为数据节点
每当我运行mapreduce流作业并且选择工作节点来运行ApplicationMaster时,该作业将在尝试连接到ResourceManager时挂起。datanode日志显示它试图连接到0.0.0.0

INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8030
INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s);
INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s);
这是
warn.resourcemanager.hostname属性的默认值。
但是,我已在Thread-site.xml中为我的两个工作节点指定了此属性:

<property>
   <name>yarn.resourcemanager.hostname</name>
   <value>hadoop-master</value>
</property>
我还通过转到hadoop-worker1:9864并访问worker节点的web界面查看加载的内容来检查配置:

<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop-master</value>
<final>false</final>
<source>yarn-site.xml</source>
</property>

我不确定该怎么办,我相信这可能与流媒体作业没有正确加载设置有关,任何帮助都将不胜感激,因为我已经在这个问题上被困了2天。

我已经将
-D纱线.resourcemanager.hostname=hadoop master
标志添加到mapred流式处理命令中,现在似乎可以工作了。

我已经将
-D纱线.resourcemanager.hostname=hadoop master
标志添加到mapred流式处理命令中,并且它现在似乎起作用了

<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop-master</value>
<final>false</final>
<source>yarn-site.xml</source>
</property>
hadoop@hadoop-worker1:/opt/hadoop$ yarn node --list
2019-06-15 18:47:42,119 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/165.22.19.161:8032
Total Nodes:3
         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
hadoop-worker2:40673            RUNNING hadoop-worker2:8042                                0
hadoop-worker1:41875            RUNNING hadoop-worker1:8042                                1
hadoop-master:40075             RUNNING hadoop-master:8042                                 0
hadoop@hadoop-worker1:/opt/hadoop$