Hadoop 在map reduce作业中洗牌/排序失败

在我的map reduce作业中，当我将reducer的数量设置为0时，程序将在不到5分钟的时间内成功完成。但是，当我将reducer的数量设置为某个值而不是0（介于1-40之间）时，甚至没有指定reducer类，它会非常缓慢地执行许多失败的任务，并突然失败，并显示以下错误消息。由于我没有指定reducer类，导致此错误的唯一原因是对映射器的输出进行排序。映射器生成大约27000个不同的关键点。这里有什么问题？日志文件中也没有任何东西可以帮助您。我唯一的猜测是大量的密钥导致了这个问题。我怎样才能解决这个问题

  13/10/28 13:06:40 INFO mapred.JobClient:  map 70% reduce 18%
  13/10/28 13:07:13 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000052_2, Status : FAILED
  Task attempt_201310101346_0661_m_000052_2 failed to report status for 600 seconds. Killing!
  13/10/28 13:07:19 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000059_2, Status : FAILED
  Task attempt_201310101346_0661_m_000059_2 failed to report status for 600 seconds. Killing!
  13/10/28 13:07:23 INFO mapred.JobClient:  map 70% reduce 19%
  13/10/28 13:07:24 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000060_2, Status : FAILED
 Task attempt_201310101346_0661_m_000060_2 failed to report status for 600 seconds. Killing!
 13/10/28 13:07:26 INFO mapred.JobClient:  map 70% reduce 20%
 13/10/28 13:07:47 INFO mapred.JobClient:  map 71% reduce 20%
 13/10/28 13:07:54 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000061_2, Status : FAILED
 Task attempt_201310101346_0661_m_000061_2 failed to report status for 600 seconds. Killing!
 13/10/28 13:08:22 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000062_2, Status : FAILED
 Task attempt_201310101346_0661_m_000062_2 failed to report status for 600 seconds. Killing!
 13/10/28 13:09:03 INFO mapred.JobClient:  map 72% reduce 20%
 13/10/28 13:09:15 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000063_2, Status : FAILED
 Task attempt_201310101346_0661_m_000063_2 failed to report status for 600 seconds. Killing!
13/10/28 13:09:18 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000066_2, Status : FAILED
 Task attempt_201310101346_0661_m_000066_2 failed to report status for 600 seconds. Killing!
13/10/28 13:09:20 INFO mapred.JobClient: Task Id : attempt_201310101346_0661_m_000068_2,     Status : FAILED
Task attempt_201310101346_0661_m_000068_2 failed to report status for 600 seconds. Killing!
13/10/28 13:09:55 INFO mapred.JobClient:  map 72% reduce 21%
13/10/28 13:10:30 INFO mapred.JobClient:  map 72% reduce 22%
13/10/28 13:10:33 INFO mapred.JobClient:  map 73% reduce 22%
13/10/28 13:12:21 INFO mapred.JobClient:  map 74% reduce 22%
13/10/28 13:13:40 INFO mapred.JobClient:  map 74% reduce 23%
13/10/28 13:14:09 INFO mapred.JobClient: Job complete: job_201310101346_0661
13/10/28 13:14:09 INFO mapred.JobClient: Counters: 31
13/10/28 13:14:09 INFO mapred.JobClient:   File System Counters
13/10/28 13:14:09 INFO mapred.JobClient:     FILE: Number of bytes read=0
13/10/28 13:14:09 INFO mapred.JobClient:     FILE: Number of bytes written=542433323
13/10/28 13:14:09 INFO mapred.JobClient:     FILE: Number of read operations=0
13/10/28 13:14:09 INFO mapred.JobClient:     FILE: Number of large read operations=0
13/10/28 13:14:09 INFO mapred.JobClient:     FILE: Number of write operations=0
13/10/28 13:14:09 INFO mapred.JobClient:     HDFS: Number of bytes read=316452240
13/10/28 13:14:09 INFO mapred.JobClient:     HDFS: Number of bytes written=0
13/10/28 13:14:09 INFO mapred.JobClient:     HDFS: Number of read operations=100
13/10/28 13:14:09 INFO mapred.JobClient:     HDFS: Number of large read operations=0
13/10/28 13:14:09 INFO mapred.JobClient:     HDFS: Number of write operations=0
13/10/28 13:14:09 INFO mapred.JobClient:   Job Counters 
13/10/28 13:14:09 INFO mapred.JobClient:     Failed map tasks=1
13/10/28 13:14:09 INFO mapred.JobClient:     Launched map tasks=169
13/10/28 13:14:09 INFO mapred.JobClient:     Launched reduce tasks=1
13/10/28 13:14:09 INFO mapred.JobClient:     Data-local map tasks=23
13/10/28 13:14:09 INFO mapred.JobClient:     Rack-local map tasks=146
13/10/28 13:14:09 INFO mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=72419457
13/10/28 13:14:09 INFO mapred.JobClient:     Total time spent by all reduces in occupied slots (ms)=1194624
13/10/28 13:14:09 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
 13/10/28 13:14:09 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
 13/10/28 13:14:09 INFO mapred.JobClient:   Map-Reduce Framework
13/10/28 13:14:09 INFO mapred.JobClient:     Map input records=6894035
13/10/28 13:14:09 INFO mapred.JobClient:     Map output records=6422195
 13/10/28 13:14:09 INFO mapred.JobClient:     Map output bytes=2300749768
13/10/28 13:14:09 INFO mapred.JobClient:     Input split bytes=6200
13/10/28 13:14:09 INFO mapred.JobClient:     Combine input records=6422195
13/10/28 13:14:09 INFO mapred.JobClient:     Combine output records=801608
13/10/28 13:14:09 INFO mapred.JobClient:     Spilled Records=801608
13/10/28 13:14:09 INFO mapred.JobClient:     CPU time spent (ms)=48077980
13/10/28 13:14:09 INFO mapred.JobClient:     Physical memory (bytes) snapshot=13246660608
13/10/28 13:14:09 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=20080971776
 13/10/28 13:14:09 INFO mapred.JobClient:     Total committed heap usage (bytes)=10181214208
 13/10/28 13:14:09 INFO mapred.JobClient:   org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter
 13/10/28 13:14:09 INFO mapred.JobClient:     BYTES_READ=316416597
 13/10/28 13:14:09 INFO mapred.JobClient: Job Failed: NA
 Exception in thread "main" java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1373)
    at Mondrian8.main(Mondrian8.java:547)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

---

当你说“日志”文件时，你是在说控制台的输出，还是去查看DataNodes上的实际映射程序日志文件？我是指存储在输出文件夹中的_log文件夹下的日志文件。