Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/16.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
如何将数据帧中的行分组到由分隔符Scala Spark分隔的单行中?_Scala_Dataframe_Apache Spark_Pyspark - Fatal编程技术网

如何将数据帧中的行分组到由分隔符Scala Spark分隔的单行中?

如何将数据帧中的行分组到由分隔符Scala Spark分隔的单行中?,scala,dataframe,apache-spark,pyspark,Scala,Dataframe,Apache Spark,Pyspark,我有Spark的数据框: +-------------+ |father|child | +-------------+ |Aaron |Adam | |Aaron |Berel | |Aaron |Kasper| |Levi |Saul | |Levi |Tiger | +-------------+ 如何按父项分组并将所有数据放在一个带分隔符的字段中 我期望的结果是: +------------------------+ |union_all_name_by_father| +--

我有Spark的数据框:

+-------------+
|father|child |
+-------------+
|Aaron |Adam  |
|Aaron |Berel |
|Aaron |Kasper|
|Levi  |Saul  |
|Levi  |Tiger |
+-------------+
如何按父项分组并将所有数据放在一个带分隔符的字段中

我期望的结果是:

+------------------------+
|union_all_name_by_father|
+------------------------+
|Aaron;Adam;Berel;Kasper |
|Levi;Saul;Tiger         |
+------------------------+

您可以使用
groupby
,然后使用
concat\ws

val df2 = df.groupBy("father").agg(
    concat_ws(";", collect_list(col("child"))).as("col2")
).select(concat_ws(";", col("father"), col("col2")).as("union_all_name_by_father"))

df2.show(false)
+------------------------+
|union_all_name_by_father|
+------------------------+
|Aaron;Adam;Berel;Kasper |
|Levi;Saul;Tiger         |
+------------------------+

在斯卡拉,我什么都没试过,我不知道该走哪条路