Hadoop 使用MR1 CDH4运行简单MapReduce流作业失败
我有一个最近从CDH3升级的运行CDH4的集群。蜂巢目前运行得很好。然而，我似乎无法让它运行甚至是简单的MR流作业（版本1）。纱线已安装但未使用。 下面是命令行输入和输出Hadoop 使用MR1 CDH4运行简单MapReduce流作业失败,hadoop,hadoop-streaming,Hadoop,Hadoop Streaming,我有一个最近从CDH3升级的运行CDH4的集群。蜂巢目前运行得很好。然而，我似乎无法让它运行甚至是简单的MR流作业（版本1）。纱线已安装但未使用。 下面是命令行输入和输出 $ /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop- streaming-2.0.0-mr1-cdh4.0.0.jar grep -input /input -output /output/ 'dfs
$ /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop- streaming-2.0.0-mr1-cdh4.0.0.jar grep -input /input -output /output/ 'dfs[a-z.]+'
packageJobJar: [/tmp/hadoop-hdfs/hadoop-unjar7491355516546899751/] [] /tmp/streamjob1375201380112960182.jar tmpDir=null
12/07/12 07:26:29 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/07/12 07:26:29 INFO util.NativeCodeLoader: Loaded the native-hadoop library
12/07/12 07:26:29 WARN snappy.LoadSnappy: Snappy native library is available
12/07/12 07:26:29 INFO snappy.LoadSnappy: Snappy native library loaded
12/07/12 07:26:29 INFO mapred.FileInputFormat: Total input paths to process : 3
12/07/12 07:26:29 INFO streaming.StreamJob: getLocalDirs(): [file:////data/hadoop-0.20/cache/mapred/mapred/local]
12/07/12 07:26:29 INFO streaming.StreamJob: Running job: job_201207120604_0018
12/07/12 07:26:29 INFO streaming.StreamJob: To kill this job, run:
12/07/12 07:26:29 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=frost:54311 -kill job_201207120604_0018
12/07/12 07:26:29 INFO streaming.StreamJob: Tracking URL: >http://alpha:50030/jobdetails.jsp?jobid=job_201207120604_0018
12/07/12 07:26:30 INFO streaming.StreamJob: map 0% reduce 0%
12/07/12 07:26:57 INFO streaming.StreamJob: map 100% reduce 100%
12/07/12 07:26:57 INFO streaming.StreamJob: To kill this job, run:
12/07/12 07:26:57 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=frost:54311 -kill job_201207120604_0018
12/07/12 07:26:57 INFO streaming.StreamJob: Tracking URL: >http://alpha:50030/jobdetails.jsp?jobid=job_201207120604_0018
12/07/12 07:26:57 ERROR streaming.StreamJob: Job not successful. Error: NA
12/07/12 07:26:57 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
2012-07-12 07:26:46,785 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201207120604_0018_m_000001_2: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:861)
at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:501)
at org.apache.hadoop.mapred.lib.IdentityMapper.map(IdentityMapper.java:38)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
您是否可以查看已提交作业的生成job.xml（通过JobTracker web界面），并查看映射器的定义值？（
mapreduce.map.class如果是这种情况，您需要查看2.0.0 hadoop的流式代码（我现在还没有准备好），以了解如何启动
hadoop-Streaming-2.0.0-mr1-cdh4.0.0.jarmapreduce.map.class如果是这种情况，您需要研究2.0.0 hadoop的流式代码（我现在还没有准备好），以了解如何使用参数启动
hadoop-streaming-2.0.0-mr1-cdh4.0.0.jar