Amazon ec2 我如何知道amazon mapreduce任务何时完成？

我正在尝试在AmazonEC2上运行mapreduce任务。 我设置了所有的配置参数，然后调用AmazonElasticMapReduce服务的runFlowJob方法。 我想知道有没有办法知道这项工作是否已经完成，情况如何。 （我需要它知道何时可以从s3中提取mapreduce结果以进行进一步处理）

当前代码只是继续执行，因为对runJobFlow的调用是非阻塞的

public void startMapReduceTask(String accessKey, String secretKey
        ,String eC2KeyPairName, String endPointURL, String jobName
        ,int numInstances, String instanceType, String placement
        ,String logDirName, String bucketName, String pigScriptName) {
    log.info("Start running MapReduce");

    // config.set
    ClientConfiguration config = new ClientConfiguration();
    AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);

    AmazonElasticMapReduce service = new AmazonElasticMapReduceClient(credentials, config);
    service.setEndpoint(endPointURL);

    JobFlowInstancesConfig conf = new JobFlowInstancesConfig();

    conf.setEc2KeyName(eC2KeyPairName);
    conf.setInstanceCount(numInstances);
    conf.setKeepJobFlowAliveWhenNoSteps(true);
    conf.setMasterInstanceType(instanceType);
    conf.setPlacement(new PlacementType(placement));
    conf.setSlaveInstanceType(instanceType);

    StepFactory stepFactory = new StepFactory();

    StepConfig enableDebugging = new StepConfig()
    .withName("Enable Debugging")
    .withActionOnFailure("TERMINATE_JOB_FLOW")
    .withHadoopJarStep(stepFactory.newEnableDebuggingStep());

    StepConfig installPig = new StepConfig()
    .withName("Install Pig")
    .withActionOnFailure("TERMINATE_JOB_FLOW")
    .withHadoopJarStep(stepFactory.newInstallPigStep());

    StepConfig runPigScript = new StepConfig()
    .withName("Run Pig Script")
    .withActionOnFailure("TERMINATE_JOB_FLOW")
    .withHadoopJarStep(stepFactory.newRunPigScriptStep("s3://" + bucketName + "/" + pigScriptName, ""));

    RunJobFlowRequest request = new RunJobFlowRequest(jobName, conf)
    .withSteps(enableDebugging, installPig, runPigScript)
    .withLogUri("s3n://" + bucketName + "/" + logDirName);

    try {
        RunJobFlowResult res = service.runJobFlow(request);
        log.info("Mapreduce job with id[" + res.getJobFlowId() + "] completed successfully");
    } catch (Exception e) {
        log.error("Caught Exception: ", e);
    }
    log.info("End running MapReduce");      
}

谢谢

---

AWS文件中的aviad：

一旦作业流完成，集群将停止，HDFS分区将丢失为防止数据丢失，请将作业流的最后一步配置为将结果存储在Amazon S3中。

它接着说：

如果

JobFlowInstancesDetail:KeepJobFlowAliveWhenNoSteps

参数设置为

TRUE

，则一旦步骤完成，作业流将转换为

等待

状态，而不是关闭

每个作业流中最多允许256个步骤

对于长时间运行的作业流，我们建议您定期存储结果

因此，似乎没有办法知道它何时完成。相反，您需要将数据保存为作业的一部分