Java 复杂类型的reducer中ReadFields()中的空缓冲区
我试图在映射器和reducer之间传递一个复杂的可写性,更具体地说是ObjectWritables的ArrayWritableJava 复杂类型的reducer中ReadFields()中的空缓冲区,java,hadoop,mapreduce,Java,Hadoop,Mapreduce,我试图在映射器和reducer之间传递一个复杂的可写性,更具体地说是ObjectWritables的ArrayWritable public class ObjectArrayWritable extends ArrayWritable { public ObjectArrayWritable() { super(ObjectWritable.class); } } 我的映射器:mapper 我的减速机:reducer reducer在可写接口的readfiel
public class ObjectArrayWritable extends ArrayWritable {
public ObjectArrayWritable() {
super(ObjectWritable.class);
}
}
我的映射器:mapper
我的减速机:reducer
reducer在可写接口的readfields()
方法上崩溃,并引发以下异常:
java.lang.NullPointerException
at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:183)
at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:66)
at org.apache.hadoop.io.ArrayWritable.readFields(ArrayWritable.java:90)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:116)
at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:175)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
当我尝试实现自己的自定义可写并调试它时,我注意到读取字段(DataInput data)
中的DataInput缓冲区接收到空值
请注意,我的错误类似于:
但是清理这个物体没有帮助
有人能帮忙吗?我不认为您正在处理ObjectArrayWritable中没有任何内容的情况。我不能确定,因为没有提供代码