Java 简单Pig脚本抛出NotSerializableException:org.apache.log4j.Level

我只是尝试执行这个愚蠢的脚本：

a = LOAD 'example';
dump a ;

其中“example”是hdfs主页中的一个文件，我得到了以下异常：

ERROR 2017: Internal error creating job configuration.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias a
        at org.apache.pig.PigServer.openIterator(PigServer.java:836)
        at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.run(Main.java:538)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias a
        at org.apache.pig.PigServer.storeEx(PigServer.java:935)
        at org.apache.pig.PigServer.store(PigServer.java:898)
        at org.apache.pig.PigServer.openIterator(PigServer.java:811)
        ... 12 more Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:848)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:294)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
        at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
        at org.apache.pig.PigServer.storeEx(PigServer.java:931)
        ... 14 more Caused by: java.io.NotSerializableException: org.apache.log4j.Level
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
        at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
        at org.apache.pig.impl.util.JarManager.createJar(JarManager.java:186)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:517)
        ... 19 more

其中第一个原因是“java.io.NotSerializableException:org.apache.log4j.Level”

我用Pig 0.11.0、0.11.1和0.12.1测试了它

Java版本“1.6.0_31”

---

有什么想法吗？

我有Hadoop：

PATH=/home/.../hadoop/bin:...

Pig不仅读取配置文件

~/hadoop/conf/core site.xml

和

~/hadoop/conf/mapred site.xml

，而且还检查

~/hadoop/lib/*

，似乎该目录中的JAR与Pig安装的JAR之间存在某种交互

---

我从

PATH

中删除了

~/hadoop/bin

，将配置文件

core site.xml

和

mapred site.xml

复制到

~/pig/conf

，一切都开始工作。

grunt是否在本地模式下运行？否，它正在群集上运行。请在本地模式下检查相同的命令好吗？使用

pig-x local运行时显示相同的异常。但我已经找到了解决办法：）谢谢！对于那些在这里寻找这篇文章的人来说，这是一个很好的选择。