Hadoop 纱线：哪个子系统连接到端口44874

我正试着在纱线上运行我的MR工作。节点3上的一个用户日志中存在此错误：

2014-10-10 00:57:16,965 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2014-10-10 00:57:16,965 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1412895371072_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@69d5af30)
2014-10-10 00:57:17,330 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2014-10-10 00:57:18,547 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node03/127.0.1.1:44874. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2014-10-10 00:57:19,548 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node03/127.0.1.1:44874. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
...
2014-10-10 00:57:27,558 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node03/127.0.1.1:44874. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2014-10-10 00:57:27,562 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node03/127.0.1.1 to node03:44874 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
        at org.apache.hadoop.ipc.Client.call(Client.java:1415)
        at org.apache.hadoop.ipc.Client.call(Client.java:1364)
        at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:231)
        at com.sun.proxy.$Proxy9.getTask(Unknown Source)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:137)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:606)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:700)
        at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1463)
        at org.apache.hadoop.ipc.Client.call(Client.java:1382)
        ... 4 more

2014-10-10 00:57:27,564 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2014-10-10 00:57:27,566 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
:

---

我在所有节点上都有相同的配置。我找不到指定端口44874的任何位置。这个错误到底说明了什么？

每当我看到Hadoop的端口出现问题时，我都会搜索端口号，看看它是否是某个端口的默认端口。你的情况似乎不是这样

据我所知，Hadoop在内部使用这种半随机端口进行某些操作，但它们无法真正配置。如果这类端口有问题，对我来说，它总是表示存在其他（可检测的）问题

我建议您再次查看所有日志以查找其他问题。同时检查namenode状态（web界面）并确保所有连接都正常工作。

如果“半随机”是指完全随机，而“无法真正配置”是指在加固环境中使用时未记录且完全失效，则正确

问题是map reduce作业使用的是动态端口。Horton当然没有记录创建随机端口的原因

到目前为止，答案是：禁用防火墙或允许在每个数据节点上设置高范围（32768-65535）。我仍在寻找这种情况发生的原因