错误:java.io.IOException:错误的值类:class org.apache.hadoop.io.Text不是类Myclass
我有我的映射器和还原器如下。但我得到了某种奇怪的例外。 我不明白为什么它会抛出这样的异常错误:java.io.IOException:错误的值类:class org.apache.hadoop.io.Text不是类Myclass,java,hadoop,elastic-map-reduce,Java,Hadoop,Elastic Map Reduce,我有我的映射器和还原器如下。但我得到了某种奇怪的例外。 我不明白为什么它会抛出这样的异常 public static class MyMapper implements Mapper<LongWritable, Text, Text, Info> { @Override public void map(LongWritable key, Text value, OutputCollector<Text, Info> output, Rep
public static class MyMapper implements Mapper<LongWritable, Text, Text, Info> {
@Override
public void map(LongWritable key, Text value,
OutputCollector<Text, Info> output, Reporter reporter)
throws IOException {
Text text = new Text("someText")
//process
output.collect(text, infoObjeject);
}
}
public static class MyReducer implements Reducer<Text, Info, Text, Text> {
@Override
public void reduce(Text key, Iterator<Info> values,
OutputCollector<Text, Text> output, Reporter reporter)
throws IOException {
String value = "xyz" //derived in some way
//process
output.collect(key, new Text(value)); //exception occurs at this line
}
}
System.out.println("Starting v14 ");
JobConf conf = new JobConf(RouteBuilderJob.class);
conf.setJobName("xyz");
String jarLocation =ClassUtil.findContainingJar(getClass());
System.out.println("path of jar file = " + jarLocation);
conf.setJarByClass(RouteBuilderJob.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(Info.class);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(Text.class);
//am i missing something here???
conf.setMapperClass(RouteBuilderJob.RouteMapper.class);
conf.setCombinerClass(RouteBuilderJob.RouteReducer.class);
conf.setReducerClass(RouteBuilderJob.RouteReducer.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
JobClient.runJob(conf);
内部信息对象(实现可写
)使用文本
@Override
public void write(DataOutput out) throws IOException {
Gson gson = new Gson();
String searlizedStr = gson.toJson(this);
Text.writeString(out, searlizedStr);
}
@Override
public void readFields(DataInput in) throws IOException {
String s = Text.readString(in);
Gson gson = new Gson();
JsonReader jsonReader = new JsonReader(new StringReader(s));
jsonReader.setLenient(true);
Info info = gson.fromJson(jsonReader, Info.class);
//set fields using this.somefield = info.getsomefield()
}
从技术上讲,reduce的输出类型应该与输入类型相同。如果使用合路器将合路器的输出馈入减速器,则必须如此。此处的“值”是什么。我找不到“value”变量的任何声明或任何东西?“output.collect(key,newtext(value));”中的line@Backtrack价值观是一种价值观string@Backtrack你能帮我吗?我花了很多时间调试这个。。我正在调查这个问题,在拆下合路器后问题就解决了@回溯谢谢回溯也帮了我。但是,为什么reducer希望这两种类型保持不变呢
@Override
public void write(DataOutput out) throws IOException {
Gson gson = new Gson();
String searlizedStr = gson.toJson(this);
Text.writeString(out, searlizedStr);
}
@Override
public void readFields(DataInput in) throws IOException {
String s = Text.readString(in);
Gson gson = new Gson();
JsonReader jsonReader = new JsonReader(new StringReader(s));
jsonReader.setLenient(true);
Info info = gson.fromJson(jsonReader, Info.class);
//set fields using this.somefield = info.getsomefield()
}