Dataframe Pyspark-基于列表中的值分解数据帧

这是我的数据框：

+--------------------+--------------------+
|             core_id|    movie_genres_upd|
+--------------------+--------------------+
|12f99f04-5168-438...|[Comedy, Mockumen...|
|32c7d12f-6bf2-4e5...|[Action, Blockbus...|
|9f067041-3b49-4db...|[Animation, Comed...|
|c6d203cb-afcf-4e8...|[Action, Adventur...|
|b02416f9-5761-48f...|[Adventure, Anima...|

id  genres
1   ["comedy", "blockbuster"]
2   ["drama", "animation", "comedy"] 

id genres
1  "comedy"
1  "blockbuster"
2  "drama"
2  "animation"
2  "comedy"

以下是我的数据类型：

[('core_id', 'string'), ('movie_genres_upd', 'array<string>')]

所需数据帧：

+--------------------+--------------------+
|             core_id|    movie_genres_upd|
+--------------------+--------------------+
|12f99f04-5168-438...|[Comedy, Mockumen...|
|32c7d12f-6bf2-4e5...|[Action, Blockbus...|
|9f067041-3b49-4db...|[Animation, Comed...|
|c6d203cb-afcf-4e8...|[Action, Adventur...|
|b02416f9-5761-48f...|[Adventure, Anima...|

id  genres
1   ["comedy", "blockbuster"]
2   ["drama", "animation", "comedy"] 

id genres
1  "comedy"
1  "blockbuster"
2  "drama"
2  "animation"
2  "comedy"

---

我是pyspark的新手，所以我正在努力解决这个问题。任何帮助都将不胜感激

如果这有帮助，请告诉我：

>>> from pyspark.sql.functions import explode
>>> from pyspark.sql.types import (
...     StringType,
...     StructField,
...     StructType,
...     ArrayType
... )
>>>
>>> schema = StructType([
...     StructField('core_id', StringType(), True),
...     StructField('movie_genres_upd', ArrayType(StringType()), True)
... ])
>>>
>>> list = [[1, ["comedy", "blockbuster"]], [2, ["drama", "animation", "comedy"]]]
>>> df = spark.createDataFrame(list, schema)
>>> df2 = df.select('core_id', explode("movie_genres_upd").alias('genre'))
>>> df2.show()
+-------+-----------+
|core_id|      genre|
+-------+-----------+
|      1|     comedy|
|      1|blockbuster|
|      2|      drama|
|      2|  animation|
|      2|     comedy|
+-------+-----------+

---

这回答了你的问题吗？