Question

我试图使用sqoop将一个小表从mysql导出到HDFS。该表有2列id（主键）和名称。我能够通过sqoop列出数据库和表。但是在将表导入HDFS时会出现异常。请帮助。以下是错误日志。

13/12/04 02:05:38 WARN conf.Configuration: session.id is deprecated. 
Instead, use   dfs.metrics.session-id
13/12/04 02:05:38 INFO jvm.JvmMetrics: 
Initializing JVM Metrics  withprocessName=JobTracker,sessionId=
13/12/04 02:05:39 INFO mapreduce.JobSubmitter: 
Cleaning up the staging area file:/tmp/hadoop-hadoop/mapred/staging/hadoop1439217057
/.staging/job_local1439217057_0001
13/12/04 02:05:39 ERROR 
security.UserGroupInformation:PriviledgedActionException  as:hadoop      
(auth:SIMPLE) 
cause:java.io.FileNotFoundException: 
File does not exist: hdfs://prat1:9000/home/hadoop/usr/sqoop-1.4.3-cdh4.3.0/lib/commons-    compress-1.4.1.jar
13/12/04 02:05:39 DEBUG util.ClassLoaderStack: 
Restoring classloader:sun.misc.Launcher$AppClassLoader@35a16869
13/12/04 02:05:39 ERROR tool.ImportTool: 
Encountered IOException running import job:  java.io.FileNotFoundException: File does not  exist:
hdfs://prat1:9000/home/hadoop/usr/
sqoop-1.4.3-cdh4.3.0/lib/commons-compress-1.4.1.jar
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:824)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:290)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:361)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1287)
    at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:173)
    at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:151)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:226)
    at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:555)
    at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:111)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

Answer 1

尝试这样做：

su root
输入root密码
su hdfs

然后运行sqoop命令，它将像冠军一样工作！

Answer 2

由于您没有写入权限，因此您将获得security.UserGroupInformation：PriviledgedActionException

尝试以hdfs用户身份登录，然后运行sqoop命令。

su root

输入密码

su hdfs

然后运行sqoop export命令。

Answer 3

授予/usr/lib/sqoop/lib/<mysqljavaconnector>-bin.jar个文件777权限，并确保/usr/lib/sqoop/*和/usr/local/hadoop/*中的文件归同一用户所有。

导入到hdfs时出现Sqoop错误

3 个答案: