Oozie工作流在Main类中失败[org.apache.oozie.action.hadoop.Hive2Main]

时间:2016-01-26 17:13:49

标签: oozie

我创建了一个简单的Oozie工作流程,该工作流失败了#34; Main class [org.apache.oozie.action.hadoop.Hive2Main]"。它本身的蜂巢声明运行良好。

我得到的错误如下



2016-01-26 09:43:52,362 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@:start:] Start action [0000826-151214145249775-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-01-26 09:43:52,363 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@:start:] [***0000826-151214145249775-oozie-oozi-W@:start:***]Action status=DONE
2016-01-26 09:43:52,363 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@:start:] [***0000826-151214145249775-oozie-oozi-W@:start:***]Action updated in DB!
2016-01-26 09:43:52,481 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] Start action [0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-01-26 09:43:53,631 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] checking action, hadoop job ID [job_1450130002590_10737] status [RUNNING]
2016-01-26 09:43:53,633 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] [***0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie***]Action status=RUNNING
2016-01-26 09:43:53,634 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] [***0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie***]Action updated in DB!
2016-01-26 09:44:22,463 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[<<host_name>>] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] callback for action [0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie]
2016-01-26 09:44:22,690 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] action completed, external ID [job_1450130002590_10737]
2016-01-26 09:44:22,697 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
2016-01-26 09:44:22,761 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@hive_load_dtv_nielsen_dma_xref_oozie] ERROR is considered as FAILED for SLA
2016-01-26 09:44:22,825 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@fail] Start action [0000826-151214145249775-oozie-oozi-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-01-26 09:44:22,825 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@fail] [***0000826-151214145249775-oozie-oozi-W@fail***]Action status=DONE
2016-01-26 09:44:22,825 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[<<host_name>>] USER[kchandr2] GROUP[-] TOKEN[] APP[dtv_nielsen_dma_xref_intr_dly_load_wf] JOB[0000826-151214145249775-oozie-oozi-W] ACTION[0000826-151214145249775-oozie-oozi-W@fail] [***0000826-151214145249775-oozie-oozi-W@fail***]Action updated in DB!
&#13;
&#13;
&#13;

workflow.xml和属性文件如下所示

&#13;
&#13;
[kchandr2@node001]/home/kchandr2/wf>vi dtv_nielsen_dma_xref_intr_dly_load_wf.properties
nameNode=hdfs://nameservice1
jobTracker=node001.abcd.com:8032
queueName=default
oozie.libpath=${nameNode}/user/oozie/share/libext
oozie.wf.application.path=${nameNode}/user/kchandr2/wf
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
hiveSiteXML=${oozie.wf.application.path}/hive-site.xml
scripts=${nameNode}/user/kchandr2/wf
workflow_name=dtv_nielsen_dma_xref_intr_dly_load_wf

db_dbname=BIHT_DB

hcat_metastore_uri=thrift://node001.abcd.com:9083
hcat_metastore_principal=hive/_HOST@BIBD-TEST.abcd.com
user_host_name=username@node001


jdbc_hive=jdbc:hive2://node002.abcd.com:10000/biht_db
password=foo

hive2_jdbc_uri=jdbc:hive2://node002.abcd.com:10000/default
hive2_server_principal=hive/node002.abcd.com@BIBD-TEST.abcd.com

[kchandr2@node001]/home/kchandr2/wf>vi workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.4" name="dtv_nielsen_dma_xref_intr_dly_load_wf">

<credentials>
<credential name="hive2_cred" type="hive2">
<property>
<name>hive2.jdbc.url</name>
<value>${hive2_jdbc_uri}</value>
</property>
<property>
<name>hive2.server.principal</name>
<value>${hive2_server_principal}</value>
</property>
</credential>
</credentials>

<start to="hive_load_dtv_nielsen_dma_xref_oozie" />

<action name="hive_load_dtv_nielsen_dma_xref_oozie" cred='hive2_cred'>
<hive2 xmlns="uri:oozie:hive2-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>${hiveSiteXML}</job-xml>
<jdbc-url>${jdbc_hive}</jdbc-url>
<password>${password}</password>
<script>${scripts}/dtv_nielsen_dma_xref_load.hql</script>
<param>db_dbname_dbname=${db_dbname}</param>
</hive2>
<ok to="dtv_nielsen_dma_xref_intr_dly_load_wf_completed"/>
<error to="fail"/>
</action>

<kill name="fail">
<message>Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>

<end name="dtv_nielsen_dma_xref_intr_dly_load_wf_completed"/>

</workflow-app>



Execution statement:-
---------------------
oozie job -oozie http://node002.abcd.com:11000/oozie -config /home/kchandr2/wf/dtv_nielsen_dma_xref_intr_dly_load_wf.properties -run -verbose > /home/kchandr2/wf/Logs/dtv_nielsen_dma_xref_intr_dly_load_wf_$(date '+%Y%m%d_%H%M%S').log 2>&1
&#13;
&#13;
&#13;

0 个答案:

没有答案