Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/django/21.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Mapreduce 如何测试以null形式发出键的映射器,即context.write(null,<;somevalue>;);_Mapreduce_Mrunit - Fatal编程技术网

Mapreduce 如何测试以null形式发出键的映射器,即context.write(null,<;somevalue>;);

Mapreduce 如何测试以null形式发出键的映射器,即context.write(null,<;somevalue>;);,mapreduce,mrunit,Mapreduce,Mrunit,我有一个mapreduce程序,只有mapper,没有设置reducer。我想测试一下。我有下面的测试代码 @Test public void testMapper() throws IOException { mapDriver.withInput(new LongWritable(0l), new Text( "af00bac654249b9d27982f19064338f4,54.0258822077885,-1.5683213346637

我有一个mapreduce程序,只有mapper,没有设置reducer。我想测试一下。我有下面的测试代码

@Test
    public void testMapper() throws IOException {

      mapDriver.withInput(new LongWritable(0l), new Text(
              "af00bac654249b9d27982f19064338f4,54.0258822077885,-1.56832133466378,20121022,105507,026542913532,2093,87"));
      mapDriver.withOutput(null, [some value]);
      mapDriver.runTest();
    }
调用mapDriver.withOutput(null,[some value]);它在异常下面抛出的这一行

java.lang.NullPointerException 位于org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:58) 位于org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:91) 位于org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:104)


注意:映射器通用签名映射器
有人能告诉我如何为发出null的映射器编写测试场景吗

如果我使用nullwriteable.get,则会得到如下异常 java.lang.NullPointerException 位于org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:73) 位于org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:91) 位于org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:104) 位于org.apache.hadoop.mrunit.TestDriver.copy(TestDriver.java:608) 位于org.apache.hadoop.mrunit.TestDriver.copyPair(TestDriver.java:612) 位于org.apache.hadoop.mrunit.TestDriver.addOutput(TestDriver.java:118) 位于org.apache.hadoop.mrunit.TestDriver.withOutput(TestDriver.java:138) 在com.gfk.gxl.etl.common.ExtractCSVTest.testMapper上(ExtractCSVTest.java:73) 在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处

看起来更像 然而,答案并不能解决我的问题

 with few more research i have below update
    class org.apache.avro.generic.GenericData$Record is not able to get serializer and deserializer
    in org.apache.hadoop.mrunit.internal.io.Serialization and both are coming as null which is causing the null pointer exception



 From API code snippet  for org.apache.hadoop.mrunit.internal.io.Serialization starting at line      no 61 to 70

  try {
      serializer = (Serializer<Object>) serializationFactory
          .getSerializer(clazz);
      deserializer = (Deserializer<Object>) serializationFactory
          .getDeserializer(clazz);
    } catch (NullPointerException e) {
      throw new IllegalStateException(
          "No applicable class implementing Serialization in conf at io.serializations for "
              + orig.getClass(), e);
    }
above method serializer \ deserializer  are coming null . do we have some way to avoid it
我在下面更新了一些研究
类org.apache.avro.generic.GenericData$Record无法获取序列化程序和反序列化程序
在org.apache.hadoop.mrunit.internal.io.Serialization中,两者都是null,这导致了null指针异常
从第61到70行开始的org.apache.hadoop.mrunit.internal.io.Serialization的API代码片段
试一试{
序列化程序=(序列化程序)序列化工厂
.getSerializer(clazz);
反序列化程序=(反序列化程序)序列化工厂
.getDeserializer(clazz);
}捕获(NullPointerException e){
抛出新的非法状态异常(
“没有在io.serializations的conf中实现序列化的适用类”
+源getClass(),e);
}
上面的方法序列化器\反序列化器为空。我们有办法避免吗

使用NullWritable.get()方法。希望这会有所帮助。

不幸的是,虽然Hadoop可以接受空键,但您现在不能在MRUnit中使用空键,MRUnit团队计划在将来支持空键,请参见此处

注意:映射器通用签名映射器您能否在此提及void的意义。您是否解决了该问题?我和你一样明白这个问题,我的制图员就像制图员一样
 with few more research i have below update
    class org.apache.avro.generic.GenericData$Record is not able to get serializer and deserializer
    in org.apache.hadoop.mrunit.internal.io.Serialization and both are coming as null which is causing the null pointer exception



 From API code snippet  for org.apache.hadoop.mrunit.internal.io.Serialization starting at line      no 61 to 70

  try {
      serializer = (Serializer<Object>) serializationFactory
          .getSerializer(clazz);
      deserializer = (Deserializer<Object>) serializationFactory
          .getDeserializer(clazz);
    } catch (NullPointerException e) {
      throw new IllegalStateException(
          "No applicable class implementing Serialization in conf at io.serializations for "
              + orig.getClass(), e);
    }
above method serializer \ deserializer  are coming null . do we have some way to avoid it