Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/400.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 自定义可写可比较文件出错_Java_Hadoop - Fatal编程技术网

Java 自定义可写可比较文件出错

Java 自定义可写可比较文件出错,java,hadoop,Java,Hadoop,我有一个类genkey,它实现了writable和comparable。我需要将此类的对象从mapper传递到reducer。制图器工作正常。现在,当我最终尝试通过context.write获取输出时,我得到以下错误 class org.apache.hadoop.io.Text不是类genkey 这是我的密码 public static class Mappertwo extends Mapper<Object, Text, genkey, IntWritable> { p

我有一个类
genkey
,它实现了
writable
comparable
。我需要将此类的对象从mapper传递到reducer。制图器工作正常。现在,当我最终尝试通过
context.write
获取输出时,我得到以下错误

class org.apache.hadoop.io.Text不是类genkey

这是我的密码

public static class Mappertwo extends Mapper<Object, Text, genkey, IntWritable> {
    private genkey ky=new genkey();
    public void map(Object key , Text value , Context context)
        throws IOException , InterruptedException{

        .............

        context.write(ky,one);
    }          
}
public static class Reducertwo extends Reducer<genkey, IntWritable, Text, IntWritable>
{              
     public void reduce(genkey ukey , Iterable<IntWritable> values , Context context)
        throws IOException , InterruptedException{          

        IntWritable result = new IntWritable();
        Text nw=new Text();
        int sum=0;

        for (IntWritable val : values){
            sum += val.get();
        }

        String s=ukey.getkey();
        nw.set(s);
        result.set(sum);
        context.write(nw,result);
        //this line here gives the error...                        }
}

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException{
    if (args.length != 5) {
        System.err.println("exceeded array limit:");
        System.exit(-1);
    }

     Configuration conf1 = new Configuration();
     Job job1= new Job(conf1, "round2");
     conf1.set("t",args[2]);
     DistributedCache.addCacheFile(new Path("/user/hduser/out/part-r-00000").toUri(),job1.getConfiguration());
     job1.setJarByClass(MapSide.class);
     job1.setMapperClass(Mappertwo.class);
     job1.setCombinerClass(Reducertwo.class);
     job1.setReducerClass(Reducertwo.class);
     job1.setOutputKeyClass(Text.class);
     job1.setOutputValueClass(IntWritable.class);
     job1.setMapOutputKeyClass(genkey.class);
     FileInputFormat.addInputPath(job1, new Path(args[0]));
     FileOutputFormat.setOutputPath(job1, new Path(args[3]));
     System.exit(job1.waitForCompletion(true)?0:1);
}
公共静态类Mappertwo扩展Mapper{
私有genkey ky=新的genkey();
公共无效映射(对象键、文本值、上下文)
抛出IOException、InterruptedException{
.............
上下文。写(ky,one);
}          
}
公共静态类Reducer两个扩展Reducer
{              
public void reduce(genkey-ukey、Iterable值、上下文)
抛出IOException,InterruptedException{
IntWritable结果=新的IntWritable();
Text nw=新文本();
整数和=0;
for(可写入值:值){
sum+=val.get();
}
字符串s=ukey.getkey();
西北偏西(s);;
结果集(总和);
write(nw,result);
//这一行给出了错误…}
}
公共静态void main(字符串[]args)引发IOException、ClassNotFoundException、InterruptedException{
如果(参数长度!=5){
System.err.println(“超出数组限制:”;
系统退出(-1);
}
Configuration conf1=新配置();
Job job1=新工作(conf1,“round2”);
conf1.set(“t”,args[2]);
DistributedCache.addCacheFile(新路径(“/user/hduser/out/part-r-00000”).toUri(),job1.getConfiguration());
job1.setJarByClass(MapSide.class);
job1.setMapperClass(Mappertwo.class);
job1.setCombinerClass(还原二级);
作业1.setReducerClass(Reducertwo.class);
job1.setOutputKeyClass(Text.class);
job1.setOutputValueClass(IntWritable.class);
job1.setMapOutputKeyClass(genkey.class);
addInputPath(作业1,新路径(args[0]);
setOutputPath(作业1,新路径(args[3]);
系统退出(作业1.等待完成(真)?0:1;
}

您定义了要写入的
文本,intwriteable
。如果要编写
genkey,则IntWritable
减速器定义必须如下所示:

public static class Reducertwo extends Reducer<genkey, IntWritable, genkey, IntWritable>
 job1.setOutputKeyClass(genkey.class);