Hadoop MapReduce作业中的InvalidAuxServiceException

在最近创建的开源hadoop集群上运行map reduce作业时，我遇到以下异常。我正在运行最新的hadoop版本3.3.0

2020-09-03 00:58:30,068 INFO mapreduce.Job: Task Id : attempt_1599094453872_0001_m_000000_2, Status : FAILED
Container launch failed for container_1599094453872_0001_01_000004 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:83)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:437)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateExceptionImpl(SerializedExceptionPBImpl.java:171)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:182)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:163)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:394)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.lang.Thread.run(Thread.java:820)

根据一些在线建议，我在warn-site.xml中添加了以下两个属性，并重新启动了warn和dfs。但是，它仍然抛出与上面相同的异常。有时作业会成功，但有例外

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

纱线.节点管理器.辅助服务
mapreduce_shuffle
warn.nodemanager.aux-services.mapreduce\u shuffle.class
org.apache.hadoop.mapred.ShuffleHandler

您是否遵循Hadoop的官方建议？根据hadoop的官方建议，我尝试将属性。我重新启动了dfs和resourcemanager。我仍然得到同样的例外。现在，map reduce作业也没有成功。下面是my-site.xml中的属性。warn.nodemanager.aux-services mapreduce_shuffle，AuxServiceFromHDFS Thread.nodemanager.aux-services.mapreduce_shuffle.class org.apache.hadoop.mapred.ShuffleHandler Thread.nodemanager.aux-services.AuxServiceFromHDFS.class org.apache.auxtest.AuxServiceFromHDFS2我已经尝试了几乎所有来自web的不同解决方案。但它仍然不起作用。我想不出我做错了什么。非常感谢您的帮助。我真的被这个问题困住了。非常感谢您的帮助或建议。