Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/377.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
错误:java.io.IOException:错误的值类:class org.apache.hadoop.io.Text不是类Myclass_Java_Hadoop_Elastic Map Reduce - Fatal编程技术网

错误:java.io.IOException:错误的值类:class org.apache.hadoop.io.Text不是类Myclass

错误:java.io.IOException:错误的值类:class org.apache.hadoop.io.Text不是类Myclass,java,hadoop,elastic-map-reduce,Java,Hadoop,Elastic Map Reduce,我有我的映射器和还原器如下。但我得到了某种奇怪的例外。 我不明白为什么它会抛出这样的异常 public static class MyMapper implements Mapper<LongWritable, Text, Text, Info> { @Override public void map(LongWritable key, Text value, OutputCollector<Text, Info> output, Rep

我有我的映射器和还原器如下。但我得到了某种奇怪的例外。 我不明白为什么它会抛出这样的异常

public static class MyMapper implements Mapper<LongWritable, Text, Text, Info> {

    @Override
    public void map(LongWritable key, Text value,
        OutputCollector<Text, Info> output, Reporter reporter)
        throws IOException {
        Text text = new Text("someText")
            //process 
        output.collect(text, infoObjeject);
    }

}

public static class MyReducer implements Reducer<Text, Info, Text, Text> {

    @Override
    public void reduce(Text key, Iterator<Info> values,
        OutputCollector<Text, Text> output, Reporter reporter)
        throws IOException {
        String value = "xyz" //derived in some way
        //process
        output.collect(key, new Text(value)); //exception occurs at this line
    }

}

System.out.println("Starting v14 ");
JobConf conf = new JobConf(RouteBuilderJob.class);
conf.setJobName("xyz");

String jarLocation =ClassUtil.findContainingJar(getClass());

System.out.println("path of jar file = " + jarLocation);

conf.setJarByClass(RouteBuilderJob.class);

conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(Info.class);

conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(Text.class);

//am i missing something here???

conf.setMapperClass(RouteBuilderJob.RouteMapper.class);
conf.setCombinerClass(RouteBuilderJob.RouteReducer.class);
conf.setReducerClass(RouteBuilderJob.RouteReducer.class);


conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);

FileInputFormat.setInputPaths(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));

JobClient.runJob(conf);
内部信息对象(实现
可写
)使用
文本

@Override
public void write(DataOutput out) throws IOException {
    Gson gson = new Gson();
    String searlizedStr = gson.toJson(this);
    Text.writeString(out, searlizedStr);
}

@Override
public void readFields(DataInput in) throws IOException {
    String s = Text.readString(in);
    Gson gson = new Gson();
    JsonReader jsonReader = new JsonReader(new StringReader(s));
    jsonReader.setLenient(true);

Info info = gson.fromJson(jsonReader, Info.class);
    //set fields using this.somefield = info.getsomefield() 
}

从技术上讲,reduce的输出类型应该与输入类型相同。如果使用合路器将合路器的输出馈入减速器,则必须如此。

此处的“值”是什么。我找不到“value”变量的任何声明或任何东西?“output.collect(key,newtext(value));”中的line@Backtrack价值观是一种价值观string@Backtrack你能帮我吗?我花了很多时间调试这个。。我正在调查这个问题,在拆下合路器后问题就解决了@回溯谢谢回溯也帮了我。但是,为什么reducer希望这两种类型保持不变呢
@Override
public void write(DataOutput out) throws IOException {
    Gson gson = new Gson();
    String searlizedStr = gson.toJson(this);
    Text.writeString(out, searlizedStr);
}

@Override
public void readFields(DataInput in) throws IOException {
    String s = Text.readString(in);
    Gson gson = new Gson();
    JsonReader jsonReader = new JsonReader(new StringReader(s));
    jsonReader.setLenient(true);

Info info = gson.fromJson(jsonReader, Info.class);
    //set fields using this.somefield = info.getsomefield() 
}