Apache spark Pyspark：从另一个结构数组创建一个结构数组

我正在使用Pyspark 2.4，希望从

df_1

创建

df_2

：

df_1:

root
 |-- request: array (nullable = false)
 |    |-- address: struct (nullable = false)
 |    |    |-- street: string (nullable  = false)
 |    |    |-- postcode: string (nullable  = false)

root
 |-- request: array (nullable = false)
 |    |-- address: struct (nullable = false)
 |    |    |-- street: string (nullable  = false)

df_2:

root
 |-- request: array (nullable = false)
 |    |-- address: struct (nullable = false)
 |    |    |-- street: string (nullable  = false)
 |    |    |-- postcode: string (nullable  = false)

root
 |-- request: array (nullable = false)
 |    |-- address: struct (nullable = false)
 |    |    |-- street: string (nullable  = false)

我知道UDF是一种方法，但是有没有其他方法，比如使用

map（）

，来实现相同的目标？

使用函数：

df_2 = df_1.withColumn("request", expr("transform(request, x -> struct(x.street) as address)"))

对于

request

数组的每个元素，我们只选择

street

字段并创建一个新结构