List Spark数据帧值到Scala列表
我有一个数据帧,其中一列具有数组:List Spark数据帧值到Scala列表,list,scala,dataframe,apache-spark,scala-collections,List,Scala,Dataframe,Apache Spark,Scala Collections,我有一个数据帧,其中一列具有数组: +----------------------------+ |User | Color | +----------------------------+ |User1 | [Green,Blue,Red] | |User2 | [Blue,Red] | +----------------------------+ 我正在尝试筛选User1,并将颜色列表放入Scala列表中: val colorsList
+----------------------------+
|User | Color |
+----------------------------+
|User1 | [Green,Blue,Red] |
|User2 | [Blue,Red] |
+----------------------------+
我正在尝试筛选User1
,并将颜色列表放入Scala列表中:
val colorsList: List[String] = List("Green","Blue","Red")
以下是我到目前为止尝试的内容(输出作为注释添加):
尝试1:
val dfTest1 = myDataframe.where("User=='User1'").select("Color").rdd.map(r => r(0)).collect()
println(dfTest1) //[Ljava.lang.Object;@44022255
for(EachColor<- dfTest1){
println(EachColor) //WrappedArray(Green, Blue, Red)
}
尝试3:
val dfTest32 = myDataframe.where("User=='User1'").select("Color").rdd.map(r => r(0)).collect.toList
println(dfTest32) //List(WrappedArray(Green, Blue, Red))
for(EachColor <- dfTest32){
println(EachColor) //WrappedArray(Green, Blue, Red)
}
您可以尝试获取as
Seq[String]
并转换toList
:
val colorsList = df.where("User=='User1'")
.select("Color")
.rdd.map(r => r.getAs[Seq[String]](0))
.collect()(0)
.toList
或同等地
val colorsList = df.where("User=='User1'")
.select("Color")
.collect()(0)
.getAs[Seq[String]](0)
.toList
val colorsList = df.where("User=='User1'")
.select("Color")
.rdd.map(r => r.getAs[Seq[String]](0))
.collect()(0)
.toList
val colorsList = df.where("User=='User1'")
.select("Color")
.collect()(0)
.getAs[Seq[String]](0)
.toList