从Java执行Pig脚本时出错
我使用以下代码从java运行pig:从Java执行Pig脚本时出错,java,hadoop,apache-pig,Java,Hadoop,Apache Pig,我使用以下代码从java运行pig: import org.apache.pig.ExecType; import org.apache.pig.PigServer; public class Pigtest { public static void main(String[] args) { try { PigServer pigServer = new PigServer(ExecType.MAPREDUCE); run
import org.apache.pig.ExecType;
import org.apache.pig.PigServer;
public class Pigtest {
public static void main(String[] args) {
try {
PigServer pigServer = new PigServer(ExecType.MAPREDUCE);
runQuery(pigServer);
}catch(Exception e) {
e.printStackTrace();
}
}
public static void runQuery(PigServer pigServer) {
try {
pigServer.registerQuery("input1 = LOAD '/pigtest.txt' as (f1:int, f2:chararray);");
System.out.println("File found!");
pigServer.registerQuery("store input1 into '/pigtest_out';");
} catch(Exception e) {
e.printStackTrace();
}
}
}
如何解决这个问题?
但它给出了以下错误:
java.io.IOException: Cannot run program "chmod": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:189)
at java.lang.ProcessImpl.start(ProcessImpl.java:133)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1021)
... 16 more
16/03/11 15:05:00 ERROR mapReduceLayer.MapReduceLauncher: Failed to produce result in: "file:/pigtest_out"
16/03/11 15:05:00 INFO mapReduceLayer.MapReduceLauncher: Failed!
java.io.IOException:无法运行程序“chmod”:CreateProcess error=2,系统找不到指定的文件
位于java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
位于org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
位于org.apache.hadoop.util.Shell.run(Shell.java:134)
位于org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
位于org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
位于org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
位于org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
位于org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
位于org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
位于org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
位于org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
位于org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
位于org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
位于org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
位于org.apache.hadoop.mapred.jobcontrol.jobcontrol.startReadyJobs(jobcontrol.java:247)
位于org.apache.hadoop.mapred.jobcontrol.jobcontrol.run(jobcontrol.java:279)
运行(Thread.java:722)
原因:java.io.IOException:CreateProcess error=2,系统找不到指定的文件
在java.lang.ProcessImpl.create(本机方法)
位于java.lang.ProcessImpl。(ProcessImpl.java:189)
在java.lang.ProcessImpl.start(ProcessImpl.java:133)
位于java.lang.ProcessBuilder.start(ProcessBuilder.java:1021)
... 还有16个
2011年3月16日15:05:00错误mapReduceLayer.MapReduceLauncher:未能在“文件:/pigtest\u out”中生成结果
16/03/11 15:05:00信息mapReduceLayer.MapReduceLauncher:失败!
上面写着:
原因:java.io.IOException:CreateProcess error=2,系统找不到指定的文件
看起来,您的输入文件/pigtest.txt在HDFS中不存在
Load和store语句期望在MAPREDUCE执行模式下使用HDFS路径。确保这些数据路径是HDFS filepath。Pig无法将输出写入/pigtest\u out。请检查您是否拥有在“/”位置创建文件的所有权限。尝试将输出放在用户主文件夹中