Java 自定义可写可比较文件出错
我有一个类Java 自定义可写可比较文件出错,java,hadoop,Java,Hadoop,我有一个类genkey,它实现了writable和comparable。我需要将此类的对象从mapper传递到reducer。制图器工作正常。现在,当我最终尝试通过context.write获取输出时,我得到以下错误 class org.apache.hadoop.io.Text不是类genkey 这是我的密码 public static class Mappertwo extends Mapper<Object, Text, genkey, IntWritable> { p
genkey
,它实现了writable
和comparable
。我需要将此类的对象从mapper传递到reducer。制图器工作正常。现在,当我最终尝试通过context.write
获取输出时,我得到以下错误
class org.apache.hadoop.io.Text不是类genkey
这是我的密码
public static class Mappertwo extends Mapper<Object, Text, genkey, IntWritable> {
private genkey ky=new genkey();
public void map(Object key , Text value , Context context)
throws IOException , InterruptedException{
.............
context.write(ky,one);
}
}
public static class Reducertwo extends Reducer<genkey, IntWritable, Text, IntWritable>
{
public void reduce(genkey ukey , Iterable<IntWritable> values , Context context)
throws IOException , InterruptedException{
IntWritable result = new IntWritable();
Text nw=new Text();
int sum=0;
for (IntWritable val : values){
sum += val.get();
}
String s=ukey.getkey();
nw.set(s);
result.set(sum);
context.write(nw,result);
//this line here gives the error... }
}
public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException{
if (args.length != 5) {
System.err.println("exceeded array limit:");
System.exit(-1);
}
Configuration conf1 = new Configuration();
Job job1= new Job(conf1, "round2");
conf1.set("t",args[2]);
DistributedCache.addCacheFile(new Path("/user/hduser/out/part-r-00000").toUri(),job1.getConfiguration());
job1.setJarByClass(MapSide.class);
job1.setMapperClass(Mappertwo.class);
job1.setCombinerClass(Reducertwo.class);
job1.setReducerClass(Reducertwo.class);
job1.setOutputKeyClass(Text.class);
job1.setOutputValueClass(IntWritable.class);
job1.setMapOutputKeyClass(genkey.class);
FileInputFormat.addInputPath(job1, new Path(args[0]));
FileOutputFormat.setOutputPath(job1, new Path(args[3]));
System.exit(job1.waitForCompletion(true)?0:1);
}
公共静态类Mappertwo扩展Mapper{
私有genkey ky=新的genkey();
公共无效映射(对象键、文本值、上下文)
抛出IOException、InterruptedException{
.............
上下文。写(ky,one);
}
}
公共静态类Reducer两个扩展Reducer
{
public void reduce(genkey-ukey、Iterable值、上下文)
抛出IOException,InterruptedException{
IntWritable结果=新的IntWritable();
Text nw=新文本();
整数和=0;
for(可写入值:值){
sum+=val.get();
}
字符串s=ukey.getkey();
西北偏西(s);;
结果集(总和);
write(nw,result);
//这一行给出了错误…}
}
公共静态void main(字符串[]args)引发IOException、ClassNotFoundException、InterruptedException{
如果(参数长度!=5){
System.err.println(“超出数组限制:”;
系统退出(-1);
}
Configuration conf1=新配置();
Job job1=新工作(conf1,“round2”);
conf1.set(“t”,args[2]);
DistributedCache.addCacheFile(新路径(“/user/hduser/out/part-r-00000”).toUri(),job1.getConfiguration());
job1.setJarByClass(MapSide.class);
job1.setMapperClass(Mappertwo.class);
job1.setCombinerClass(还原二级);
作业1.setReducerClass(Reducertwo.class);
job1.setOutputKeyClass(Text.class);
job1.setOutputValueClass(IntWritable.class);
job1.setMapOutputKeyClass(genkey.class);
addInputPath(作业1,新路径(args[0]);
setOutputPath(作业1,新路径(args[3]);
系统退出(作业1.等待完成(真)?0:1;
}
您定义了要写入的文本,intwriteable
。如果要编写genkey,则IntWritable
减速器定义必须如下所示:
public static class Reducertwo extends Reducer<genkey, IntWritable, genkey, IntWritable>
job1.setOutputKeyClass(genkey.class);