Java ApachePig:无法运行我自己的Pig.jar和Pig-withouthadoop.jar

Java ApachePig:无法运行我自己的Pig.jar和Pig-withouthadoop.jar,java,hadoop,mapreduce,apache-pig,Java,Hadoop,Mapreduce,Apache Pig,我有一个运行Hadoop 0.20.2和Pig 0.10的集群。 我想在Pig的源代码中添加一些日志,并在集群上运行我自己的Pig版本 我所做的: 使用“ant”命令生成项目 有pig.jar和pig-withouthadoop.jar 将jar复制到集群的namenode上的Pig home目录 执行任务 然后我得到了以下std输出: 2013-03-25 06:35:05,226 [main] WARN org.apache.pig.backend.hadoop20.PigJobContr

我有一个运行Hadoop 0.20.2和Pig 0.10的集群。 我想在Pig的源代码中添加一些日志,并在集群上运行我自己的Pig版本

我所做的:

  • 使用“ant”命令生成项目

  • 有pig.jar和pig-withouthadoop.jar

  • 将jar复制到集群的namenode上的Pig home目录

  • 执行任务

  • 然后我得到了以下std输出:

    2013-03-25 06:35:05,226 [main] WARN  org.apache.pig.backend.hadoop20.PigJobControl -   falling back to default JobControl (not using hadoop 0.20 ?)
    java.lang.NoSuchFieldException: runnerState
        at java.lang.Class.getDeclaredField(Class.java:1882)
        at org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
        at  org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:97)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
        at org.apache.pig.PigServer.launchPlan(PigServer.java:1320)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1305)
        at org.apache.pig.PigServer.execute(PigServer.java:1295)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:480)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
    2013-03-25 06:35:05,229 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
    2013-03-25 06:35:05,260 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
    2013-03-25 06:35:05,272 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
    2013-03-25 06:35:06,041 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job9091543475518322185.jar
    2013-03-25 06:35:10,974 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job9091543475518322185.jar created
    2013-03-25 06:35:10,995 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
    2013-03-25 06:35:11,006 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
    2013-03-25 06:35:11,006 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
    2013-03-25 06:35:11,006 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
    2013-03-25 06:35:11,181 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
    
    出了什么问题?除了替换namenode的安装目录中的pig.jar和pig-withouthadoop.jar之外,我还应该做些什么吗


    帮助…

    我忽略了一点:pig-withouthadoop.jar应该使用特定的Hadoop版本编译。 我用以下方法编译了jar,它成功了:

    % ant clean jar-withouthadoop -Dhadoopversion=23
    

    如果我运行的是hadoop-1.0.4版本,我应该如何编译?我应该把hadoop-core-1.0.4.jar复制到$PIG_HOME吗?我已经导出-->导出PIG\u HOME=/HOME/hadoop/apache/PIG-0.11.1;导出路径=$PATH:$PIG_HOME/bin;export PIG_CLASSPATH=/home/hadoop/apache/hadoop-1.0.4/conf;
    % ant clean jar-withouthadoop -Dhadoopversion=23