从JavaPairRDD转换为数据集时的Java Spark异常

时间:2018-06-21 20:43:25

标签: java scala apache-spark dataframe dataset

我正在尝试将JavaPairRDD转换为数据集。这是代码:

Encoder<Tuple2<String, Tuple2<Integer,String>>> encoder2 =
                 Encoders.tuple(Encoders.STRING(), Encoders.tuple(Encoders.INT(),Encoders.STRING()));
        Dataset<Row> userViolationsDetails = spark.createDataset(JavaPairRDD.toRDD(tempMaxRDD2),encoder2).toDF("value1","value2");
        Dataset<Row> tempMax2 = userViolationsDetails.selectExpr("value1", "value2._1 as value2", "value2._2 as value3");

运行项目时,出现此异常:

Exception in thread "main" org.apache.spark.sql.AnalysisException: No such struct field _1 in value, value; line 1 pos 0

如何解决此问题?

0 个答案:

没有答案