将WrappedArray转换为DataFrame中的字符串

Question

我有一个DataFrame，如下所示：

+---+--------------------------------+---+--------------------------------+---+--------------------------------+
| _1|                              _2| _3|                              _4| _5|                              _6|
+---+--------------------------------+---+--------------------------------+---+--------------------------------+
|  2|WrappedArray(String,String,St...|  1|WrappedArray(String,String,St...|  0|WrappedArray(String,String,St...|
|  3|WrappedArray(String,String,St...|  1|WrappedArray(String,String,St...|  0|WrappedArray(String,String,St...|
|  3|WrappedArray(String,String,St...|  2|WrappedArray(String,String,St...|  1|WrappedArray(String,String,St...|
|  3|WrappedArray(String,String,St...|  2|WrappedArray(String,String,St...|  0|WrappedArray(String,String,St...|
+---+--------------------------------+---+--------------------------------+---+--------------------------------+

我想将WrappedArray转换为字符串并具有以下输出：

+---+--------------------+---+--------------------+---+--------------------+
| _1|                  _2| _3|                  _4| _5|                  _6|
+---+--------------------+---+--------------------+---+--------------------+
|  2| String,String,St...|  1| String,String,St...|  0| String,String,St...|
|  3| String,String,St...|  1| String,String,St...|  0| String,String,St...|
|  3| String,String,St...|  2| String,String,St...|  1| String,String,St...|
|  3| String,String,St...|  2| String,String,St...|  0| String,String,St...|
+---+--------------------+---+--------------------+---+--------------------+

如何使用Scala在Spark中进行转换？

Answer 1

您可以使用concat_ws函数从数组创建字符串为

val finalDf = df.withColumn("_2", concat_ws(",", $"_2"))
  .withColumn("_4", concat_ws(",", $"_4"))
  .withColumn("_6", concat_ws(",", $"_6"))

希望这会有所帮助！