Amazon web services Amazon EMR--等待步骤

我正在编写一个运行aws emr命令的bash脚本（aws emr版本1.5.2）

如何让脚本等到emr作业完成后再继续？

--等待步骤

选项现在已贬值

via我得到了这个，但这似乎是错误的方法：

STEP_STATUS_STATE=$(aws emr list-steps --cluster-id ${CLUSTER_ID} | jq '.Steps[0].Status.State' | tr -d '"')
while [[ ${STEP_STATUS_STATE} == PENDING ]] || [[ ${STEP_STATUS_STATE} == RUNNING ]]; do
    STEP_STATUS_STATE=$(aws emr list-steps --cluster-id ${CLUSTER_ID} | jq '.Steps[0].Status.State' | tr -d '"')
    echo $(date) ${STEP_STATUS_STATE}
    sleep 10
done

---

我使用AWS java api等待作业完成，如下所示。希望这有帮助

 public static final List<JobFlowExecutionState> DONE_STATES = Arrays
        .asList(new JobFlowExecutionState[] {
                JobFlowExecutionState.COMPLETED,
                JobFlowExecutionState.FAILED,
                JobFlowExecutionState.TERMINATED });

---

尝试使用AWS emr等待步骤完成选项。

---

有一个参考。

很有趣-但是我希望我可以通过一个简单的命令行参数来实现这一点，就像以前版本的aws允许的那样…对于持久化EMR集群来说，这不是一个好的实践。您应该让群集公共DNS主节点调用您的命令。类似于“每10分钟请求一次DNS值，一旦DNS不为空，就获取它”。

  public static boolean isDone(String value) {
    JobFlowExecutionState state = JobFlowExecutionState.fromValue(value);
    return Constants.DONE_STATES.contains(state);
}

   .
   .
  STATUS_LOOP: while (true) {
        DescribeJobFlowsRequest desc = new DescribeJobFlowsRequest(
                Arrays.asList(new String[] { result.getJobFlowId() }));
        DescribeJobFlowsResult descResult = emr.describeJobFlows(desc);
        for (JobFlowDetail detail : descResult.getJobFlows()) {
            String state = detail.getExecutionStatusDetail().getState();
            if (isDone(state)) {
                logger.info("Job " + state + ": " + detail.toString());

                if(loadToDailyDB && state.equalsIgnoreCase("COMPLETED"))
                {

                    //Do something
                }
                if(!state.equalsIgnoreCase("COMPLETED"))
                {

                }

                break STATUS_LOOP;
            } else if (!lastState.equals(state)) {
                lastState = state;
                logger.info("Job " + state + " at "
                        + new Date().toString());
            }
        }
        Thread.sleep(75000);