Warning: file_get_contents(/data/phpspider/zhask/data//catemap/6/apache/8.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/search/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache hadoop mapreduce common friends reducer溢出_Apache_Hadoop - Fatal编程技术网

Apache hadoop mapreduce common friends reducer溢出

Apache hadoop mapreduce common friends reducer溢出,apache,hadoop,Apache,Hadoop,我正在尝试运行下面的代码,寻找两个人之间的普通朋友。输入如下 A : B C D B : A C D E C : A B D E D : A B C E E : B C D 我无法在输出文件中获得任何输出,也没有例外 请在下面找到我的代码 public class Friend { public static class Mapperfriend extends Mapper<Object, Text, Text, Text>{ private

我正在尝试运行下面的代码,寻找两个人之间的普通朋友。输入如下

A : B C D

B : A C D E

C : A B D E

D : A B C E

E : B C D
我无法在输出文件中获得任何输出,也没有例外

请在下面找到我的代码

public class Friend {
    public static class Mapperfriend extends Mapper<Object, Text, Text, Text>{


        private Text  vendor = new Text();


        @Override
        protected void map(Object key, Text value, Context context) throws IOException, InterruptedException {
        StringTokenizer tokenizer = new StringTokenizer(value.toString(), "\n");
        String line = null;
        String[] lineArray = null;
        String[] friendArray = null;
        String[] tempArray = null;    



        while(tokenizer.hasMoreTokens()){
        line = tokenizer.nextToken();
        lineArray = line.split(":");
        friendArray = lineArray[1].split(" ");
        tempArray = new String[2];
        for(int i = 0; i < friendArray.length; i++){
                tempArray[0] = friendArray[i];
                tempArray[1] = lineArray[0];
                Arrays.sort(tempArray);
                context.write(new Text(tempArray[0] + " " + tempArray[1]), new Text(lineArray[1]));
        }

        }


    }}

    public static class ReducerFriend extends Reducer<Text,Text,Text,Text>{

        @Override
        protected void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException {


       Text[] texts = new Text[2];
                int index = 0;


                for(Text val: values)
                {
                        texts[index++] = new Text(val);
                }
                String[] list1 = texts[0].toString().split(" ");
                String[] list2 = texts[1].toString().split(" ");
                List<String> list = new LinkedList<String>();
                for(String friend1 : list1){
                        for(String friend2 : list2){
                                if(friend1.equals(friend2)){
                                        list.add(friend1);
                                }
                        }
                }
                StringBuffer sb = new StringBuffer();
                for(int i = 0; i < list.size(); i++){
                        sb.append(list.get(i));
                        if(i != list.size() - 1)
                                sb.append(" ");
                }
                context.write(key, new Text(sb.toString()));
        } 
    }




    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws Exception {
        // TODO code application logic here

        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf, "Friends");
        job.setJarByClass(Friend.class);
        job.setMapperClass(Mapperfriend.class);
        job.setReducerClass(ReducerFriend.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }



}
公共类好友{
公共静态类Mapperfriend扩展映射器{
私有文本供应商=新文本();
@凌驾
受保护的空映射(对象键、文本值、上下文)引发IOException、InterruptedException{
StringTokenizer tokenizer=新的StringTokenizer(value.toString(),“\n”);
字符串行=null;
字符串[]lineArray=null;
字符串[]friendArray=null;
字符串[]tempArray=null;
while(tokenizer.hasMoreTokens()){
line=tokenizer.nextToken();
lineArray=直线分割(“:”);
friendArray=lineArray[1]。拆分(“”);
tempArray=新字符串[2];
for(int i=0;i
映射器类从第一个类发出整个键。而不是拾取阵列。拆下后,效果良好