Hadoop:跟踪本地作业的进度

Hadoop:跟踪本地作业的进度,hadoop,Hadoop,我目前正在尝试做类似的事情。不同之处在于,我的目标作业是本地作业(这是在单元测试中运行的)。以下是我正在做的事情(我对mapProgress感兴趣): 我认为JobClient不是为了获取本地工作的信息而设计的。有没有一种方法可以为本地运行的作业执行类似的操作 final String job = this.config.get("mapreduce.job.id"); final JobClient client = new JobClient(this.con

我目前正在尝试做类似的事情。不同之处在于,我的目标作业是本地作业(这是在单元测试中运行的)。以下是我正在做的事情(我对mapProgress感兴趣):

我认为JobClient不是为了获取本地工作的信息而设计的。有没有一种方法可以为本地运行的作业执行类似的操作

        final String job = this.config.get("mapreduce.job.id");
        final JobClient client = new JobClient(this.config);
        final RunningJob running = client.getJob(JobID.forName(job));
        if (running == null) {
            Logger.warn(
                this, String.format("Job id does not exist: %s", job)
            );
            throw new IllegalStateException(
                String.format("No running job found: %s", job)
            );
        }
        while (running.mapProgress() < 1.0) {
            TimeUnit.SECONDS.sleep(1);
        }
[←[0;33mWARN←[m] Thread-6 org.apache.hadoop.mapred.LocalJobRunner: job_local204744704_0002
java.lang.Exception: java.lang.IllegalStateException: No running job found: job_local204744704_0002
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.lang.IllegalStateException: No running job found: job_local204744704_0002
        at com.xockets.hadoop.XsInput.run(XsInput.java:150)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)