Hadoop auxService:mapreduce_shuffle在配置单元上不存在

Hadoop auxService:mapreduce_shuffle在配置单元上不存在,hadoop,hive,hdfs,yarn,Hadoop,Hive,Hdfs,Yarn,我正在使用Hive1.2.0和Hadoop2.6.0。每当我在我的机器上运行蜂巢时。。。select query工作正常,但在出现count(*)的情况下,它会显示以下错误: 此任务的诊断消息:的容器启动失败 集装箱_14346464658807 _0001 _01 000005: org.apache.hadoop.warn.exceptions.InvalidAuxServiceException:auxService:mapreduce\u shuffle不存在 sun.reflect.N

我正在使用Hive1.2.0和Hadoop2.6.0。每当我在我的机器上运行蜂巢时。。。select query工作正常,但在出现
count(*)
的情况下,它会显示以下错误:

此任务的诊断消息:的容器启动失败 集装箱_14346464658807 _0001 _01 000005: org.apache.hadoop.warn.exceptions.InvalidAuxServiceException:auxService:mapreduce\u shuffle不存在 sun.reflect.NativeConstructorAccessorImpl.newInstance0(本机方法) 在 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 在 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 位于java.lang.reflect.Constructor.newInstance(Constructor.java:422) 在 org.apache.hadoop.warn.api.records.impl.pb.SerializedExceptionPBImpl.InstanceeException(SerializedExceptionPBImpl.java:168) 在 org.apache.hadoop.warn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106) 在 org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155) 在 org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 运行(Thread.java:745)

失败:执行错误,从返回代码2 org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce作业已启动: 阶段1:映射:1减少:1 HDFS读取:0 HDFS写入:0失败 花费的MapReduce CPU总时间:0毫秒


请在warn-site.xml中使用此选项;当您将框架设置为用作纱线时,它开始查找这些值

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>

纱线.节点管理器.辅助服务
mapreduce_shuffle
warn.nodemanager.aux-services.mapreduce\u shuffle.class
org.apache.hadoop.mapred.ShuffleHandler

当您将框架设置为用作纱线时,它开始查找这些值。对于Java Map reduce,我也面临同样的错误。当我设置如下配置时:

Configuration conf = new Configuration();
conf.set("mapreduce.framework.name", "yarn");

Container launch failed for container_1448364218966_0001_01_000002 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
    at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
我通过设置以下内容来解决此问题:

<configuration>

<!-- Site specific YARN configuration properties -->


<property>
               <name>yarn.nodemanager.aux-services</name>
               <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

</configuration>

纱线.节点管理器.辅助服务
mapreduce_shuffle
warn.nodemanager.aux-services.mapreduce\u shuffle.class
org.apache.hadoop.mapred.ShuffleHandler

~

应该是warn.nodemanager.aux-services.mapreduce\u shuffle.class?我使用了warn.nodemanager.aux-services.mapreduce\u shuffle.class,它成功了@AravindKrishnakumar你是在那之后重新启动蜂巢还是?@Anil你是在那之后重新启动蜂巢还是?你是在那之后重新启动蜂巢还是执行了哪些步骤?