Sorting 执行辅助排序时获取空指针异常

我只是hadoop的初学者。我在执行第二次排序时遇到空指针异常

这是我的mapper类

这是我的groupo comparator类

这就是我犯的错误

public class SecondarySortComparator extends WritableComparator {


    public SecondarySortComparator(){

        super(Employee.class);
        System.out.println("sort");
    }
    @Override
    public int compare(WritableComparable a, WritableComparable b) {
        // TODO Auto-generated method stub
        Employee employee1 = (Employee)a;
        Employee employee2 = (Employee)b;
        int i = employee1.getEmployeeSalary()>employee2.getEmployeeSalary()?1:-1;
        return i;
    }

13/09/01 19:13:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/09/01 19:13:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/09/01 19:13:47 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/09/01 19:13:47 INFO mapred.FileInputFormat: Total input paths to process : 1
13/09/01 19:13:47 INFO mapred.JobClient: Running job: job_local_0001
13/09/01 19:13:47 INFO util.ProcessTree: setsid exited with exit code 0
13/09/01 19:13:47 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1b3f8f6
13/09/01 19:13:47 INFO mapred.MapTask: numReduceTasks: 1
13/09/01 19:13:47 INFO mapred.MapTask: io.sort.mb = 100
13/09/01 19:13:48 INFO mapred.JobClient:  map 0% reduce 0%
13/09/01 19:13:48 INFO mapred.MapTask: data buffer = 79691776/99614720
sort13/09/01 19:13:48 INFO mapred.MapTask: record buffer = 262144/327680

1
1
1
1
13/09/01 19:13:49 INFO mapred.MapTask: Starting flush of map output
13/09/01 19:13:49 WARN mapred.LocalJobRunner: job_local_0001
java.lang.NullPointerException
    at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:96)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
    at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
    at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
13/09/01 19:13:49 INFO mapred.JobClient: Job complete: job_local_0001
13/09/01 19:13:49 INFO mapred.JobClient: Counters: 0
13/09/01 19:13:49 INFO mapred.JobClient: Job Failed: NA
Exception in thread "main" java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
    at secondarysort.JobRunner.main(JobRunner.java:31)

对如何解决这个问题有什么建议吗

---

提前感谢

这一行似乎是问题的根源

context.write(new Text(employeeName),  employee);

您将employee对象（类型为employee）作为值而不是键发出，并且SecondarySortComparator和SecondarySortGroupingComparator都处理键而不是值

因此，主要的问题是您将文本作为键传递，这导致了问题。您可以考虑将Effice对象传递为键，而不是用于实际工作的两个比较器的文本。

您可能还希望在Employee类中放置一个默认构造函数-

public Employee() { }

---

它解决了你的问题吗？@jTheRocker，我让我的映射器将employee作为键，在sort comparator中，我将根据员工工资进行排序，仍然在org.apache.hadoop.io.WritableComparator.compare（WritableComparator.java:96）org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare中得到这个Exception（MapTask.java:1111）在org.apache.hadoop.util.QuickSort.sortInternal（QuickSort.java:70）在org.apache.hadoop.util.QuickSort.sort（QuickSort.java:59）在org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill（MapTask.java:1399）在atorg.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush（MapTask.java:1298）上在org.apache.hadoop.mapred.MapTask.runOldMapper（MapTask.java:437）在org.apache.hadoop.mapred.MapTask.run（MapTask.java:372）在org.apache.hadoop.mapred.LocalJobRunner$Job.run（LocalJobRunner.java:212）上13/09/03 23:17:48 INFO mapred.JobClient:Job complete:Job_local_0001 13/09/03 23:17:48 INFO mapred.JobClient:Counters:013/09/03 23:17:48 INFO mapred.JobClient:Job Failed:NA线程“main”java.io.IOException:Job Failed！位于org.apache.hadoop.mapred.JobClient.runner.runner.runJob（JobClient.java:1265）secondarysort.JobRunner.main（JobRunner.java:31）

public class SecondarySortGroupingComparator extends WritableComparator{

public SecondarySortGroupingComparator(){

    super(Employee.class,true);
    System.out.println("group");

}
@Override
    public int compare(WritableComparable a, WritableComparable b) {
        // TODO Auto-generated method stub
        Employee employee1 = (Employee)a;
        Employee employee2 = (Employee)b;
        return employee1.getEmployeeName().compareTo(employee2.getEmployeeName());
    }

}

13/09/01 19:13:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/09/01 19:13:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/09/01 19:13:47 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/09/01 19:13:47 INFO mapred.FileInputFormat: Total input paths to process : 1
13/09/01 19:13:47 INFO mapred.JobClient: Running job: job_local_0001
13/09/01 19:13:47 INFO util.ProcessTree: setsid exited with exit code 0
13/09/01 19:13:47 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1b3f8f6
13/09/01 19:13:47 INFO mapred.MapTask: numReduceTasks: 1
13/09/01 19:13:47 INFO mapred.MapTask: io.sort.mb = 100
13/09/01 19:13:48 INFO mapred.JobClient:  map 0% reduce 0%
13/09/01 19:13:48 INFO mapred.MapTask: data buffer = 79691776/99614720
sort13/09/01 19:13:48 INFO mapred.MapTask: record buffer = 262144/327680

1
1
1
1
13/09/01 19:13:49 INFO mapred.MapTask: Starting flush of map output
13/09/01 19:13:49 WARN mapred.LocalJobRunner: job_local_0001
java.lang.NullPointerException
    at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:96)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
    at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
    at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
13/09/01 19:13:49 INFO mapred.JobClient: Job complete: job_local_0001
13/09/01 19:13:49 INFO mapred.JobClient: Counters: 0
13/09/01 19:13:49 INFO mapred.JobClient: Job Failed: NA
Exception in thread "main" java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
    at secondarysort.JobRunner.main(JobRunner.java:31)

context.write(new Text(employeeName),  employee);

public Employee() { }