Hadoop 2 mapreduce作业在提交后挂起

我试图在EC2集群上运行hadoop dictcp，但提交后作业挂起。有人知道问题的原因吗？谢谢

"2014-09-16 03:04:09,386 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
2014-09-16 03:04:09,502 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
2014-09-16 03:04:10,557 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06' - Unexpected response code 404, expected 200
2014-09-16 03:04:10,575 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06_%24folder%24' - Unexpected response code 404, expected 200
2014-09-16 03:04:10,797 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06' - Unexpected response code 404, expected 200
2014-09-16 03:04:10,955 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06_%24folder%24' - Unexpected response code 404, expected 200
2014-09-16 03:04:11,319 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06' - Unexpected response code 404, expected 200
2014-09-16 03:04:11,337 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06_%24folder%24' - Unexpected response code 404, expected 200
2014-09-16 03:04:11,395 WARN  httpclient.RestS3Service (RestS3Service.java:performRequest(393)) - Response '/olap_log%2Flog%2Fprod%2Fs3_tracking_log_csv%2Fstat_clicks%2F1_12%2F883%2F2014%2F06' - Unexpected response code 404, expected 200
2014-09-16 03:04:13,265 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
2014-09-16 03:04:13,265 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
2014-09-16 03:04:14,285 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
2014-09-16 03:04:14,285 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
2014-09-16 03:04:15,412 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(368)) - number of splits:21
2014-09-16 03:04:16,114 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.jar is deprecated. Instead, use mapreduce.job.jar
2014-09-16 03:04:16,116 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2014-09-16 03:04:16,116 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2014-09-16 03:04:16,117 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
2014-09-16 03:04:16,117 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2014-09-16 03:04:16,117 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.job.name is deprecated. Instead, use mapreduce.job.name
2014-09-16 03:04:16,117 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
2014-09-16 03:04:16,118 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2014-09-16 03:04:16,118 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2014-09-16 03:04:16,118 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2014-09-16 03:04:16,119 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
2014-09-16 03:04:16,119 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(824)) - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
2014-09-16 03:04:16,587 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(438)) - Submitting tokens for job: job_1410832828185_0009
2014-09-16 03:04:17,592 INFO  client.YarnClientImpl (YarnClientImpl.java:submitApplication(124)) - Submitted application application_1410832828185_0009 to ResourceManager at /10.120.109.238:8032
2014-09-16 03:04:17,632 INFO  mapreduce.Job (Job.java:submit(1222)) - The url to track the job: http://ip-10-120-109-238.ec2.internal:8088/proxy/application_1410832828185_0009/
2014-09-16 03:04:17,632 INFO  tools.DistCp (DistCp.java:execute(164)) - DistCp job-id: job_1410832828185_0009
2014-09-16 03:04:17,633 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1267)) - Running job: job_1410832828185_0009"

My-site.xml： {

Thread.nodemanager.container-executor.classorg.apache.hadoop.Thread.server.nodemanager.DefaultContainerExecutor
纱线.nodemanager.aux-servicesmapreduce_shuffle
纱线.nodemanager.resource.memory-mb64000
纱线.调度程序.最小分配-mb2048
纱线.nodemanager.aux-services.mapreduce.shuffle.classorg.apache.hadoop.mapred.ShuffleHandler
纱线.资源管理器.资源跟踪器.地址10.120.109.238:8031
纱线.资源管理器.调度程序.地址10.120.109.238:8030
纱线.资源经理.地址10.120.109.238:8032
warn.resourcemanager.hostnameec2-54-234-24-96.compute-1.amazonaws.com
}
mapred-site.xml：
{
mapreduce.framework.nameshorn
fs.default.name
hdfs://ec2-54-234-24-96.compute-1.amazonaws.com:9000
mapred.job.tracker
ec2-54-234-24-96.compute-1.amazonaws.com:9001
mapred.map.tasks
4.
mapred.reduce.tasks
4.
mapred.tasktracker.map.tasks.max
4.
mapred.tasktracker.reduce.tasks.max
4.
mapred.output.committer.classorg.apache.hadoop.mapred.DirectFileOutputCommitter
mapreduce.reduce.java.opts-Xmx6144m
mapreduce.map.java.opts-Xmx3072m
mapreduce.reduce.shuffle.parallelcopies32
mapreduce.map.memory.mb4096
mapreduce.map.memory.mb8192
}
core-site.xml：
{
hadoop.tmp.dir
/mnt/短期hdfs
fs.default.name
hdfs://ec2-54-234-24-96.compute-1.amazonaws.com:9000
io.file.buffer.size
65536
dfs.client.read.shortcircuit
假的
dfs.client.read.shortcircuit.skip.checksum
假的
dfs.domain.socket.path
/var/run/hadoop hdfs/dn.\u端口
dfs.client.file-block-storage-locations.timeout
3000
fs.tachyon.impl
tachyon.hadoop.TFS
}

从提供的信息中很难判断，但我确实在您的输出中看到了这一点：

    Response '/olap_log/log/prod/s3_tracking_log_csv/stat_clicks/1_12/883/2014/06' - Unexpected response code 404, expected 200

该作业在尝试获取该文件时出现404（未找到）错误，我认为它是您的输入文件。也许这就是问题所在

如果您看一下，是否还有其他信息？

谢谢。我可以在hadoop 1集群上运行相同的“hadoop distcp”命令，并且没有问题下载相同的s3文件集。无法打开作业跟踪器URL。

    Response '/olap_log/log/prod/s3_tracking_log_csv/stat_clicks/1_12/883/2014/06' - Unexpected response code 404, expected 200