hadoop reduce>;减少

hadoop reduce>;减少,hadoop,mapreduce,ipc,reduce,Hadoop,Mapreduce,Ipc,Reduce,我正在hadoop上运行一个作业。这些例子效果很好 然而,我的工作是停留在地图100%减少67%。映射减少任务状态为“减少>减少” 我的reduce()方法: 你也可以分享你的驱动程序和映射程序代码!我在我原来的帖子中添加了我的map方法。但是我不明白你说的司机是什么意思?你是说提交工作的主要方法吗? public static class MyReducer extends Reducer<DoubleWritable, Text, DoubleWritable, Text> {

我正在hadoop上运行一个作业。这些例子效果很好

然而,我的工作是停留在地图100%减少67%。映射减少任务状态为“减少>减少”

我的reduce()方法:


你也可以分享你的驱动程序和映射程序代码!我在我原来的帖子中添加了我的map方法。但是我不明白你说的司机是什么意思?你是说提交工作的主要方法吗?
public static class MyReducer extends Reducer<DoubleWritable, Text, DoubleWritable, Text> {

    public void reduce(DoubleWritable key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
        Text res = new Text("");

        for(Text value : values){
            if(!value.equals("")){
                res.set(res.toString() + value.toString() + " ");
            }
        }
        context.write(key, res);
    }
}
public static class TokenizerMapper extends Mapper<Object, Text, DoubleWritable, Text>{

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException{
         StringTokenizer input = new StringTokenizer(value.toString());

         input.nextToken();
         Text page = new Text(input.nextToken());

         DoubleWritable vue = new DoubleWritable(Double.parseDouble(input.nextToken()) );
         context.write(vue, page);
    }
}
2014-10-27 10:15:37,284 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:37,285 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #614
2014-10-27 10:15:37,287 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #614
2014-10-27 10:15:37,288 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 3
2014-10-27 10:15:40,296 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:40,296 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #615
2014-10-27 10:15:40,298 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #615
2014-10-27 10:15:40,298 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 2