我有一个嵌套的json数据,其中包含要提取和构造Scala Map的嵌套字段。
这里有示例JSON:
"nested_field": [
{
"airport": "sfo",
"score": 1.0
},
{
"airport": "phx",
"score": 1.0
},
{
"airport": "sjc",
"score": 1.0
}
]
我想使用saveToES()并构造一个Scala映射,以如下所示将字段索引到ES索引中:
"nested_field": {
"properties": {
"score": {
"type": "double"
},
"airport": {
"type": "keyword",
"ignore_above": 1024
}
}
}
使用spark.read.json(“ example.json”)将json文件读入数据帧。在这种情况下构造Scala Map的正确方法是什么?
感谢您的帮助!
答案 0 :(得分:0)
您可以使用以下示例代码完成
import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods.parse
case class AirPortScores(airport: String, score: Double)
case class JsonRulesHandler(airports: List[AirPortScores])
val jsonString: String = """{"airports":[{"airport":"sfo","score":1},{"airport":"phx","score":1},{"airport":"sjc","score":1}]}"""
def loadJsonString(JsonString: String): JsonRulesHandler = {
implicit val formats: DefaultFormats.type = org.json4s.DefaultFormats
parse(JsonString).extract[JsonRulesHandler]
}
val parsedJson: JsonRulesHandler = loadJsonString(jsonString)
parsedJson.airports.foreach(println)//you can select parsedJson.airport or scores
//below ouput
AirPortScores(sfo,1.0)
AirPortScores(phx,1.0)
AirPortScores(sjc,1.0)