Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/5/fortran/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 清理(上下文)方法做什么?_Java_Hadoop_Mapreduce_Bigdata - Fatal编程技术网

Java 清理(上下文)方法做什么?

Java 清理(上下文)方法做什么?,java,hadoop,mapreduce,bigdata,Java,Hadoop,Mapreduce,Bigdata,我无法理解Hadoop中的cleanup方法到底是做什么的,以及它是如何工作的?我有下面的Map Reduce代码来计算一组数字的max、min和mean public class Statistics { public static class Map extends Mapper<LongWritable, Text, Text, Text> { public void map(LongWritable key, Text value, Cont

我无法理解Hadoop中的cleanup方法到底是做什么的,以及它是如何工作的?我有下面的Map Reduce代码来计算一组数字的max、min和mean

public class Statistics 
{
    public static class Map extends Mapper<LongWritable, Text, Text, Text>
    {
        public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException 
        {
            /* code to calculate min, max, and mean from among a bunch of numbers */
        }
        public void cleanup(Context context) throws IOException, InterruptedException
        {
            Text key_min = new Text();
            key_min.set("min");
            Text value_min = new Text();
            value_min.set(String.valueOf(min));
            context.write(key_min,value_min);

            Text key_max = new Text();
            key_max.set("max");
            Text value_max = new Text();
            value_max.set(String.valueOf(max));
            context.write(key_max,value_max);

            Text key_avg = new Text();
            key_avg.set("avg");
            Text value_avg = new Text();
            value_avg.set(String.valueOf(linear_sum)+","+count);
            context.write(key_avg,value_avg);

            Text key_stddev = new Text();
            key_stddev.set("stddev");
            Text value_stddev = new Text();
            value_stddev.set(String.valueOf(linear_sum)+","+count+","+String.valueOf(quadratic_sum));
            context.write(key_stddev,value_stddev);
        }
    }
    public static class Reduce extends Reducer<Text,Text,Text,Text>
    {
        public void reduce(Text key, Iterable<Text> values,Context context) throws IOException, InterruptedException 
        {
            /* code to further find min, max and mean from among the outputs of different mappers */
        }
    }
    public static void main(String[] args) throws Exception 
    {
        /* driver program */
    }
}
公共类统计信息
{
公共静态类映射扩展映射器
{
公共void映射(LongWritable键、文本值、上下文上下文)引发IOException、InterruptedException
{
/*从一组数字中计算最小值、最大值和平均值的代码*/
}
公共无效清除(上下文上下文上下文)引发IOException、InterruptedException
{
Text key_min=新文本();
按键最小设置(“最小”);
文本值_min=新文本();
value_min.set(String.valueOf(min));
写(key\u min,value\u min);
Text key_max=新文本();
键最大设置(“最大”);
文本值_max=新文本();
value_max.set(String.valueOf(max));
write(key\u max,value\u max);
Text key_avg=新文本();
键平均设置(“平均”);
文本值_avg=新文本();
值平均集(字符串值(线性和)+“,”+计数);
编写(键平均值、值平均值);
Text key_stddev=新文本();
按键标准设定(“标准设定”);
文本值_stddev=新文本();
value_stddev.set(String.valueOf(linear_sum)+“,“+count+”,“+String.valueOf(quadratic_sum));
write(key\u stddev,value\u stddev);
}
}
公共静态类Reduce扩展Reducer
{
公共void reduce(文本键、Iterable值、上下文上下文)引发IOException、InterruptedException
{
/*从不同映射器的输出中进一步查找最小值、最大值和平均值的代码*/
}
}
公共静态void main(字符串[]args)引发异常
{
/*驱动程序*/
}
}

那么,
cleanup(Context-Context)
方法在这里到底做了什么呢?我假设它从一堆映射器中收集输出(键、值)对,并将其传递给reducer。在其他网站上,我读到MapReduce运行的顺序是:安装->地图->清理,然后是安装->减少->清理。为什么这个程序没有使用设置方法?

这些值必须不是在Mapper中计算的,而是在Reduce step上计算的。