Java 如何多次重复映射/减少任务?

Java 如何多次重复映射/减少任务?,java,hadoop,mapreduce,Java,Hadoop,Mapreduce,为了多次重复相同的map/reduce任务,我应该将循环放在哪里?我知道它应该在主程序中,但我不知道它应该在runJob附近还是其他地方?这里是一个很好的示例,可以帮助您完成您正在尝试做的事情,这是我不久前看到的: while (counter > 0) { // reuse the conf reference with a fresh object conf = new Configuration(); // set the depth into the config

为了多次重复相同的map/reduce任务,我应该将循环放在哪里?我知道它应该在主程序中,但我不知道它应该在
runJob
附近还是其他地方?

这里是一个很好的示例,可以帮助您完成您正在尝试做的事情,这是我不久前看到的:

while (counter > 0) {
   // reuse the conf reference with a fresh object
   conf = new Configuration();
   // set the depth into the configuration
   conf.set("recursion.depth", depth + "");
   job = new Job(conf);
   job.setJobName("Graph explorer " + depth);

   job.setMapperClass(ExplorationMapper.class);
   job.setReducerClass(ExplorationReducer.class);
   job.setJarByClass(ExplorationMapper.class);
   // always work on the path of the previous depth
   in = new Path("files/graph-exploration/depth_" + (depth - 1) + "/");
   out = new Path("files/graph-exploration/depth_" + depth);

   SequenceFileInputFormat.addInputPath(job, in);
   // delete the outputpath if already exists
   if (fs.exists(out))
    fs.delete(out, true);

   SequenceFileOutputFormat.setOutputPath(job, out);
   job.setInputFormatClass(SequenceFileInputFormat.class);
   job.setOutputFormatClass(SequenceFileOutputFormat.class);
   job.setOutputKeyClass(LongWritable.class);
   job.setOutputValueClass(VertexWritable.class);
   // wait for completion and update the counter
   job.waitForCompletion(true);
   depth++;
   counter = job.getCounters().findCounter(ExplorationReducer.UpdateCounter.UPDATED)
     .getValue();
}