在hive中发出select查询时获取异常'java.lang.IllegalArgumentException（无法从空字符串创建路径）'

时间：2017-03-28 08:34:27

标签： hadoop hive hiveql

我是这个网站的新手，实际上我遇到了以下问题，导致“无法从空字符串创建路径”错误。我的hadoop版本是2.5.2，蜂巢版本是0.13.1。以下是我使用的文件详细信息和查询，任何人都可以帮我解决此问题。

文件名：tweet_info.json 文件中的数据如下：

{
    "user": {
        "lang": "en",
        "profile_background_image_url": "http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.png",
        "id_str": "568286862",
        "default_profile_image": false,
        "statuses_count": 595,
        "profile_link_color": "0084B4",
        "favourites_count": 5,
        "profile_image_url_https": "https:\/\/si0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
        "following": null,
        "profile_background_color": "C0DEED",
        "description": "Nobody can ever affect you unless you allow yourself to be affected.",
        "notifications": null,
        "profile_background_tile": false,
        "time_zone": null,
        "profile_sidebar_fill_color": "DDEEF6",
        "listed_count": 0,
        "contributors_enabled": false,
        "geo_enabled": false,
        "created_at": "Tue May 01 13:29:22 +0000 2012",
        "screen_name": "Chin_Hean",
        "follow_request_sent": null,
        "profile_sidebar_border_color": "C0DEED",
        "protected": false,
        "url": null,
        "default_profile": true,
        "name": "\u0106hr\u00ed\u0161",
        "is_translator": false,
        "show_all_inline_media": false,
        "verified": false,
        "profile_use_background_image": true,
        "followers_count": 37,
        "profile_image_url": "http:\/\/a0.twimg.com\/profile_images\/2206373396\/image_normal.jpg",
        "id": 568286862,
        "profile_background_image_url_https": "https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.png",
        "utc_offset": null,
        "friends_count": 65,
        "profile_text_color": "333333",
        "location": "Stark Industries"
    }
}

我在Hive中创建表格查询：

CREATE TABLE tweet_info(
user struct<lang: string, id: int, statuses_count: int, favourites_count: int, description: string, 
screen_name: string, url: string, friends_count: int, followers_count: int, location: string>)

ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

我的选择查询：

SELECT user.id UserId, user.screen_name UserName, user.location UserLocation, user.description UserDescription, user.lang UserLanguage, user.friends_count UserFrndsCount from tweet_info;

错误日志：

总工作数= 1 从1开始工作1 减少任务的数量设置为0，因为没有减少运算符

java.lang.IllegalArgumentException: Can not create a Path from an empty string
    at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
    at org.apache.hadoop.fs.Path.<init>(Path.java:135)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:212)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
    at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
    at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Job Submission failed with exception 'java.lang.IllegalArgumentException(Can not create a Path from an empty string)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

1 个答案:

答案 0 :(得分：0)

我终于找到了解决问题的方法。以下是我所做的更改，解决了我的所有错误。

  1. 我用hive-serdes-1.0-SNAPSHOT.jar
    2. 替换了我的serde
  2. 放置 将上面的jar放入我的hive / lib文件夹中，如下所示：
    ＆＃39; /usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-SNAPSHOT.jar'
  3. 将jar添加到我的类路径中：添加jar /usr/lib/hive/apache-hive-0.13.1-bin/lib/hive-serdes-1.0-SNAPSHOT.jar
  4. 我将ROW FORMAT SERDE改为了com.cloudera.hive.serde.JSONSerDe＆＃39; 在我的创建表查询。

    5. 这解决了我的所有错误，最后得到了我的输出。

    由于

相关问题
最新问题