Question

我想为这个json对象创建一个hive表

{"query":{"results":{"Result":[{"Phone":"(787) 812-3300","Distance":"6.62","MapUrl":"http://local.yahoo.com/info-159703323-petsmart-ponce?viewtype=map","xmlns":"urn:yahoo:lcl","Categories":null,"Title":"Petsmart","City":"Ponce","BusinessClickUrl":"http://stores.petsmart.com/Store/Details/2308","ClickUrl":"http://local.yahoo.com/info-159703323-petsmart-ponce","id":"159703323","BusinessUrl":"http://stores.petsmart.com/Store/Details/2308","State":"PR","Url":"http://local.yahoo.com/info-159703323-petsmart-ponce","Rating":{"LastReviewIntro":null,"TotalReviews":"0","TotalRatings":"0","LastReviewDate":null,"AverageRating":"NaN"},"Address":"2643 Ponce Bypass, Ste 148","Latitude":"17.994451","Longitude":"-66.638763"},{"Phone":"(787) 878-8800","Distance":"35.88","MapUrl":"http://local.yahoo.com/info-159700558-petsmart-hatillo?viewtype=map","xmlns":"urn:yahoo:lcl","Categories":null,"Title":"Petsmart","City":"Hatillo","BusinessClickUrl":"http://stores.petsmart.com/Store/Details/2272","ClickUrl":"http://local.yahoo.com/info-159700558-petsmart-hatillo","id":"159700558","BusinessUrl":"http://stores.petsmart.com/Store/Details/2272","State":"PR","Url":"http://local.yahoo.com/info-159700558-petsmart-hatillo","Rating":{"LastReviewIntro":null,"TotalReviews":"0","TotalRatings":"0","LastReviewDate":null,"AverageRating":"NaN"},"Address":"506 Truncado St. Hatillo, Puerto Rico 00659","Latitude":"18.483111","Longitude":"-66.822868"},{"Phone":"(787) 286-2999","Distance":"37.65","MapUrl":"http://local.yahoo.com/info-159704175-petsmart-caguas?viewtype=map","xmlns":"urn:yahoo:lcl","Categories":null,"Title":"Petsmart","City":"Caguas","BusinessClickUrl":"http://stores.petsmart.com/Store/Details/2188","ClickUrl":"http://local.yahoo.com/info-159704175-petsmart-caguas","id":"159704175","BusinessUrl":"http://stores.petsmart.com/Store/Details/2188","State":"PR","Url":"http://local.yahoo.com/info-159704175-petsmart-caguas","Rating":{"LastReviewIntro":null,"TotalReviews":"0","TotalRatings":"0","LastReviewDate":null,"AverageRating":"NaN"},"Address":"200 Avenida Rafael Cordero","Latitude":"18.244964","Longitude":"-66.019486"},{"Phone":"(787) 282-2966","Distance":"39.18","MapUrl":"http://local.yahoo.com/info-159725166-petsmart-san-juan?viewtype=map","xmlns":"urn:yahoo:lcl","Categories":null,"Title":"Petsmart","City":"San Juan","BusinessClickUrl":"http://stores.petsmart.com/Store/Details/2162","ClickUrl":"http://local.yahoo.com/info-159725166-petsmart-san-juan","id":"159725166","BusinessUrl":"http://stores.petsmart.com/Store/Details/2162","State":"PR","Url":"http://local.yahoo.com/info-159725166-petsmart-san-juan","Rating":{"LastReviewIntro":null,"TotalReviews":"0","TotalRatings":"0","LastReviewDate":null,"AverageRating":"NaN"},"Address":"Senorial Plaza Mall South San Juan, Puerto Rico 00926","Latitude":"18.37046","Longitude":"-66.067658"},{"Phone":"(787) 774-1370","Distance":"39.17","MapUrl":"http://local.yahoo.com/info-159699705-petsmart-guaynabo?viewtype=map","xmlns":"urn:yahoo:lcl","Categories":null,"Title":"Petsmart","City":"Guaynabo","BusinessClickUrl":"http://stores.petsmart.com/Store/Details/2160","ClickUrl":"http://local.yahoo.com/info-159699705-petsmart-guaynabo","id":"159699705","BusinessUrl":"http://stores.petsmart.com/Store/Details/2160","State":"PR","Url":"http://local.yahoo.com/info-159699705-petsmart-guaynabo","Rating":{"LastReviewIntro":null,"TotalReviews":"0","TotalRatings":"0","LastReviewDate":null,"AverageRating":"NaN"},"Address":"A2 Calle Ebano","Latitude":"18.408851","Longitude":"-66.100197"}]},"count":5,"created":"2014-02-03T05:31:18Z","lang":"en-US"}}

我的创建表格查询是

CREATE EXTERNAL TABLE data (
query STRUCT<
        results: STRUCT<
                Result: ARRAY<
                        STRUCT<
                                Phone: STRING,
                                Distance: STRING,
                                MapUrl: STRING,
                                xmlns: STRING,
                                Categories: STRING,
                                Title: STRING,
                                City: STRING,
                                BusinessClickUrl: STRING,
                                ClickUrl: STRING,
                                id: STRING,
                                BusinessUrl: STRING,
                                State: STRING,
                                Url: STRING,
                                Rating: STRUCT<
                                        LastReviewIntro: STRING,
                                        TotalReviews: STRING,
                                        TotalRatings: STRING,
                                        LastReviewDate: STRING,
                                        AverageRating: STRING
                                >,
                                Address: STRING,
                                Latitude: STRING,
                                Longitude: STRING
                        >
                >
        >,
        count: STRING,
        created: STRING,
        lang: STRING
>
) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' LOCATION '/user/hive/warehouse/test/';

但是当我说select * from data时结果列为null。

{"results":{"result":null},"count":"5","created":"2014-02-03T05:31:18Z","lang":"en-US"}
Time taken: 0.127 seconds, Fetched: 1 row(s)

我也尝试了不同的东西，比如

CREATE EXTERNAL TABLE data (
query string
) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' LOCATION '/user/hive/warehouse/test/';

对于此select *语句将显示所有结果。但是我的上述查询有什么问题？

我也试过了select get_json_object('data.query', '$') from data;这也取了空。

Answer 1

可能是因为Categories是null而不是字符串。也许是JsonSerDe可能更宽容的情况。

Answer 2

我回答，我们应该使用不同的jar来获取所有值。

 ADD JAR /hive/Hive/hive-0.12.0/lib/hive-serdes-1.0-SNAPSHOT.jar;

并且代码中的所有内容都相同，但ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'除外。我们将在嵌套块中获取所有值。

从hive表中检索数据时的空值

2 个答案: