sqoop incremental import重新启动VM

时间:2016-09-20 10:00:36

标签: sqoop

我使用以下sqoop命令将MySQL数据导入hive。

MySQL表:

create table student(id int,name varchar(20),rank int,state varchar(20),primary key(id));
 id  | name | rank | state     |
+-----+------+------+-----------+
| 100 | AAA  |    5 | AP        |
| 101 | BBB  |    2 | AP        |
| 102 | CCC  |    1 | Karnataka |
| 103 | DDD  |   10 | Karnataka |
| 104 | EEE  |    2 | AP        |
| 105 | FFF  |    3 | AP        |
| 106 | GGGG |    4 | AP        |
| 107 | HHH  |    5 | AP        |
| 108 | IIII |   15 | Karnataka |
| 109 | JJJJ |    8 | Karnataka 


sqoop import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student --hive-import --fields-terminated-by "," --lines-terminated-by "\n" --warehouse-dir /user/hive/warehouse/test.db

sqoop成功将数据导入test.db/student目录。我为该地点创建了一个蜂巢学生表。

我在MySQL学生表中添加了一些行,如

id  | name | rank | state     |
+-----+------+------+-----------+
| 100 | AAA  |    5 | AP        |
| 101 | BBB  |    2 | AP        |
| 102 | CCC  |    1 | Karnataka |
| 103 | DDD  |   10 | Karnataka |
| 104 | EEE  |    2 | AP        |
| 105 | FFF  |    3 | AP        |
| 106 | GGGG |    4 | AP        |
| 107 | HHH  |    5 | AP        |
| 108 | IIII |   15 | Karnataka |
| 109 | JJJJ |    8 | Karnataka |
| 110 | KKK  |    6 | Karnataka |
| 111 | LLLL |    7 | AP        |
| 112 | MMM  |    9 | AP 

为了导入新插入的行,我使用sqoop命令作为

sqoop import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student  --fields-terminated-by "," --lines-terminated-by "\n" --warehouse-dir /user/hive/warehouse/test.db --incremental append --check-column id --last-value 109

但是这个sqoop命令正在重启我的VM。有什么问题?我无法找到原因。

我为同一个命令创建了一个作业,

sqoop job --create import-student1 -- import --connect jdbc:mysql://localhost/test --username root --password hadoop --table student --incremental append --check-column id --last-value 109 --warehouse-dir /user/hive/warehouse/test.db --fields-terminated-by "," --lines-terminated-by "\n" --direct

sqoop job --exec import-student1

然后显示已成功完成,但数据未导入

0 个答案:

没有答案