由于“JA009:没有这样的文件或目录”,OOZIE失败

时间:2016-10-10 12:38:24

标签: hadoop hdfs oozie cloudera-cdh

我正在尝试使用oozie,hadoop和oozie版本在Hadoop中运行作业:

[root@10-0-87-4 /]# hadoop version
Hadoop 2.6.0-cdh5.8.0
Subversion http://github.com/cloudera/hadoop -r 042da8b868a212c843bcbf3594519dd26e816e79
Compiled by jenkins on 2016-07-12T23:02Z
Compiled with protoc 2.5.0
From source with checksum 2b6c319ecc19f118d6e1c823175717b5
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.8.0.jar

[root@10-0-87-4 /]# oozie version
Oozie client build version: 3.3.2.1.3.2.0-111

oozie不是来自CDH回购的原生,我通过以下方式安装:

$ wget -nv http://public-repo-1.hortonworks.com/HDP/centos6/1.x/updates/1.3.2.0/hdp.repo -O /etc/yum.repos.d/hdp.repo
$ yum install -y oozie extjs-2.2-1 --disablerepo cloudera-cdh5

这是我需要它的工作方式,请不要建议从CDH5安装本机oozie,因为我需要这个特定的oozie

当我想要运行某项工作时,问题就出现了,我收到了这个错误:

/var/log/oozie/oozie.log:2016-10-10 12:22:03,245  WARN ActionStartXCommand:542 - SERVER[] USER[u_1813] GROUP[-] TOKEN[] APP[execute-wf] JOB[0000000-161010120015197-oozie-oozi-W] ACTION[0000000-161010120015197-oozie-oozi-W@run-luigi] Error starting action [run-luigi]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: No such file or directory]
/var/log/oozie/oozie.log:2016-10-10 12:22:04,275  WARN CoordActionUpdateXCommand:542 - SERVER[] USER[u_1813] GROUP[-] TOKEN[] APP[execute-wf] JOB[0000000-161010120015197-oozie-oozi-W] ACTION[0000000-161010120015197-oozie-oozi-W@run-luigi] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

我检查了工作流程,所有提到的文件都位于HDFS中,

  <file>python.sh#python.sh</file>
  <file>oozie-luigi-pig-workflow.py#oozie-luigi-pig-workflow.py</file>
  <file>/user/oozie/share/private_key#private_key</file>
  <file>hdfs://hdfs.<some-link>:8020/user/u_1813/oozie/workflows/pig-execute/lib/file-s3distcp.jar#file-s3distcp.jar</file>
  <file>hdfs://hdfs.<some-link>:8020/user/u_1813/oozie/workflows/pig-execute/lib/file_udfs.rb#file_udfs.rb</file>

Namenode \ Datanode \ Tasktracker \ Jobtracker \ PIG-正在运行没有任何问题,唯一的问题是Oozie。

来自/var/log/oozie/oozie.log

的额外日志
2016-10-17 18:05:45,148  WARN UserGroupInformation:1696 - SERVER[] PriviledgedActionException as:u_1813 (auth:PROXY) via oozie (auth:SIMPLE) cause:ENOENT: No such file or directory
2016-10-17 18:05:45,148  WARN UserGroupInformation:1696 - SERVER[] PriviledgedActionException as:u_1813 (auth:PROXY) via oozie (auth:SIMPLE) cause:ENOENT: No such file or directory
2016-10-17 18:05:45,148  WARN ActionStartXCommand:542 - SERVER[] USER[u_1813] GROUP[-] TOKEN[] APP[execute-wf] JOB[0000000-161017180107504-oozie-oozi-W] ACTION[0000000-161017180107504-oozie-oozi-W@prepare-script] Error starting action [prepare-script]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: No such file or directory]
org.apache.oozie.action.ActionExecutorException: JA009: No such file or directory
    at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:418)
    at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:396)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:762)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:913)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:211)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:59)
    at org.apache.oozie.command.XCommand.call(XCommand.java:277)
    at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: ENOENT: No such file or directory
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:708)
    at org.apache.hadoop.fs.ChecksumFileSystem$1.apply(ChecksumFileSystem.java:500)
    at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:481)
    at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:497)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:611)
    at org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:94)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:99)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:194)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:573)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
    at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:747)
    ... 8 more
BTW,Oozie正在使用postgresql-server并且它正在工作,我可以访问Oozie UI并且可以看到Jobs和ext ..问题在我尝试运行任何类型的Jobs时开始。

你知道我该如何解决这个问题吗?

谢谢, EddieM

0 个答案:

没有答案
相关问题