Question

我使用以下代码从java运行pig：

import org.apache.pig.ExecType;
import org.apache.pig.PigServer;
public class Pigtest {
    public static void main(String[] args) {
        try {
            PigServer pigServer = new PigServer(ExecType.MAPREDUCE);
            runQuery(pigServer);
        }catch(Exception e) {
            e.printStackTrace();
        }
    }
    public static void runQuery(PigServer pigServer) {
        try {

            pigServer.registerQuery("input1 = LOAD '/pigtest.txt' as (f1:int, f2:chararray);");
            System.out.println("File found!");

            pigServer.registerQuery("store input1 into '/pigtest_out';");
        } catch(Exception e) {
            e.printStackTrace();
        }
    }
}

如何解决这个问题？但它给出了以下错误：

java.io.IOException: Cannot run program "chmod": CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
    at org.apache.hadoop.util.Shell.run(Shell.java:134)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
    at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
    at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
    at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730)
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:189)
    at java.lang.ProcessImpl.start(ProcessImpl.java:133)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1021)
    ... 16 more

16/03/11 15:05:00 ERROR mapReduceLayer.MapReduceLauncher: Failed to produce result in: "file:/pigtest_out"
16/03/11 15:05:00 INFO mapReduceLayer.MapReduceLauncher: Failed!

Answer 1

它说：

Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified

看起来，您的输入文件 /pigtest.txt 在HDFS中不存在。

加载和存储语句需要MAPREDUCE执行模式下的HDFS路径。确保这些数据路径是HDFS文件路径。

Answer 2

Pig无法将输出写入/ pigtest_out。请检查您是否拥有在＆＃34; /＆＃34;中创建文件的所有权利。地点。尝试将输出放在用户主文件夹中。

从Java执行Pig脚本时出错

2 个答案: