Sqoop导出到TD错误,将实体null放置到TimelineServer

时间:2018-10-30 12:23:50

标签: hadoop hdfs sqoop

我试图编写一个sqoop作业,以将数据从hdfs导出到Teradata。执行作业时,在映射阶段失败,并显示以下错误。我以为它的空值异常,所以我试图只包含很少的记录而没有空值的一列。仍然我有同样的错误。因此,我怀疑这与我使用的配置或jar文件有关。您能帮我指出这个问题吗?

谢谢。

脚本:

#!/bin/bash
export USER=$UserName
export PWD=$Pwd
export CLASSPATH=$CLASSPATH:/opt/dev/sha/TD2HDFS/terajdbc4.jar:/opt/dev/sha/TD2HDFS/tdgssconfig.jar:/opt/teradata/client/16.10/tbuild/jar/teradata-connector-1.4.4.jar
export HADOOP_CLASSPATH=$CLASSPATH
export CONN=jdbc:teradata://<IPAddress>/DATABASE=wrk_dv_dcl_infa_wrk_db,CHARSET=ASCII,TMODE=DEFAULT,TNANO=0

sqoop export \
  --connect $CONN --username $USER --password $PWD \
  --driver com.teradata.jdbc.TeraDriver \
  --table test_export \
  --export-dir /apps-data/sit/sha/test_export \
  --batch

控制台输出:

18/10/26 10:43:10 INFO mapreduce.Job:  map 0% reduce 0%  
18/10/26 10:43:18 INFO mapreduce.Job:  map 100% reduce 0%  
18/10/26 10:43:19 INFO mapreduce.Job: Job job_1539322671497_2141 failed with state FAILED due to: Task failed task_1539322671497_2141_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0  

系统日志:

2018-10-26 10:43:10,004 ERROR [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Error putting entity null to TimelineServer  
org.apache.hadoop.yarn.exceptions.YarnException: Incomplete entity without entity id/type  
                at org.apache.hadoop.yarn.client.api.impl.TimelineWriter.putEntities(TimelineWriter.java:88)  
                at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:348)  
                at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processEventForTimelineServer(JobHistoryEventHandler.java:1000)  
                at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:571)  
                at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:319)  
                at java.lang.Thread.run(Thread.java:748)  
2018-10-26 10:43:10,450 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0

2018-10-26 10:43:17,516 INFO [IPC Server handler 2 on 40452] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1539322671497_2141_m_683896232476677 given task: attempt_1539322671497_2141_m_000003_0  
2018-10-26 10:43:17,757 FATAL [IPC Server handler 10 on 40452]    org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1539322671497_2141_m_000000_0 - exited : java.lang.NullPointerException  
                at com.teradata.tdgss.jtdgss.TdgssConfigApi.GetMechanisms(Unknown Source)  
                at com.teradata.tdgss.jtdgss.TdgssManager.<init>(Unknown Source)  
                at com.teradata.tdgss.jtdgss.TdgssManager.<clinit>(Unknown Source)  
                at com.teradata.jdbc.jdbc.GenericTeraEncrypt.getGSSM(GenericTeraEncrypt.java:577)

0 个答案:

没有答案