Hadoop MapReduce无法连接到ResourceManager_Hadoop_Mapreduce_Hdfs_Hadoop3

Hadoop MapReduce无法连接到ResourceManager

hadoop mapreduce

Hadoop MapReduce无法连接到ResourceManager,hadoop,mapreduce,hdfs,hadoop3,Hadoop,Mapreduce,Hdfs,Hadoop3,我正在尝试使用一个单节点集群Psuedo来安装Hadoop，并使用它来完成这一任务。现在我正在尝试运行MapReduce作业，并使用它提供的示例bin/hadoop jar share/hadoop/MapReduce/hadoop-MapReduce-examples-3.0.0-alpha3.jar grep输入输出'dfs[a-z]+' 在网上查找此问题时，其他有此问题的人似乎都有此问题，但MapReduce没有。我的hdfs-site.xml与指南中提到的相同： <configur

我正在尝试使用一个单节点集群Psuedo来安装Hadoop，并使用它来完成这一任务。现在我正在尝试运行MapReduce作业，并使用它提供的示例bin/hadoop jar share/hadoop/MapReduce/hadoop-MapReduce-examples-3.0.0-alpha3.jar grep输入输出'dfs[a-z]+'

在网上查找此问题时，其他有此问题的人似乎都有此问题，但MapReduce没有。我的hdfs-site.xml与指南中提到的相同：

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

感谢您的帮助

编辑：我查看了hadoop-hadoop-resourcemanager-hadoop.log，发现：

xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8032: starting
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to active state
xxxx-xx-xx xx:xx:xx,xxx INFO org.eclipse.jetty.util.log: Logging initialized @7307ms
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/*
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
xxxx-xx-xx xx:xx:xx,xxx FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.ExceptionInInitializerError
    at com.google.inject.internal.cglib.reflect.$FastClassEmitter.<init>(FastClassEmitter.java:67)
    at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
    ...

Edit2：这是my Thread-site.xml，如果它有帮助的话：

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
    </property>
</configuration>

我正在使用Java9，目前还没有对Hadoop的Java9支持

运行jps命令表明ResourceManager和Nodemanager未启动。请查看资源管理器日志以了解确切原因？我尝试使用中的指令设置单个群集，但始终有效。试一试Hadoop 1不是这样吗？我正在使用Hadoop 3I在资源管理器日志中发现：FATAL org.apache.Hadoop.warn.server.resourcemanager.resourcemanager:Error starting resourcemanager我将在问题中添加更多日志您的warn-site.xml应该格式不正确。请检查一下

xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8032: starting
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to active state
xxxx-xx-xx xx:xx:xx,xxx INFO org.eclipse.jetty.util.log: Logging initialized @7307ms
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/*
xxxx-xx-xx xx:xx:xx,xxx INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
xxxx-xx-xx xx:xx:xx,xxx FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.ExceptionInInitializerError
    at com.google.inject.internal.cglib.reflect.$FastClassEmitter.<init>(FastClassEmitter.java:67)
    at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
    ...

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
    </property>
</configuration>