Scala 将带行的WrappedArray列转换为带行的Dataframe

我想在下面的dataframe中将包含两行的WrappedArray元素提取到包含相同行数的dataframe中

传递给printArray函数的整个数据集df如下：

+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|SECURITY_ID|combined_list                                                                                                                                                                                                                                    |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|290X2      |[[290X2, 3FA34789, 0800TS, BOXXBU, BOXXMP, 0101, 5279, 290X2, 18063, P, , 0, 0], [290X2, 3FA34782, 0800TS, BOXXBU, BOXXMP, 0102, 5322, 290X2, -863, N, , 0, 0], [290X2, 3FA34789, 0800TS, BOXXBU, BOXXMP, 0101, 5279, 290X2, -108926, N, , 0, 0]]|
|35G71      |[[35G71, 92115301, 08036C, BOXXBU, BOXXMP, 0154, 8380, 35G71, 8003, P, , 0, 0], [35G71, 92115302, 08036C, BOXXBU, BOXXMP, 0144, 8382, 35G71, -2883, N, , 0, 0]]                                                                                  |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

    Security: 290X2
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
SECURITY_ID|ACCOUNT_NO |COSTCENTER|    BU|   MPU|LONG_IND|SHORT_IND|QUANTITY|POS_NEG_QUANTITY|PROCESSED|ALLOC_QUANTITY|NET_QUANTITY|
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
290X2      |3FA34789|    0800TS|BOXXBU|BOXXMP|    0101|     5279|      18063|               P|         |             0|           0|
290X2      |3FA34782|    0800TS|BOXXBU|BOXXMP|    0102|     5322|       -863|               N|         |             0|           0|
290X2      |3FA34789|    0800TS|BOXXBU|BOXXMP|    0101|     5279|    -108926|               N|         |             0|           0|
+-------------+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+-------------+

    Security: 35G71
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
SECURITY_ID|ACCOUNT_NO |COSTCENTER|    BU|   MPU|LONG_IND|SHORT_IND|QUANTITY|POS_NEG_QUANTITY|PROCESSED|ALLOC_QUANTITY|NET_QUANTITY|
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
35G71|92115301   |    08036C|BOXXBU|BOXXMP|    0154|     8380      |8003    |               P|         |             0|           0|
35G71|92115302   |    08036C|BOXXBU|BOXXMP|    0144|     8382      |   -2883|               N|         |             0|           0|
+-------------+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+-------------+

我希望每个证券的结果如下：

+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|SECURITY_ID|combined_list                                                                                                                                                                                                                                    |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|290X2      |[[290X2, 3FA34789, 0800TS, BOXXBU, BOXXMP, 0101, 5279, 290X2, 18063, P, , 0, 0], [290X2, 3FA34782, 0800TS, BOXXBU, BOXXMP, 0102, 5322, 290X2, -863, N, , 0, 0], [290X2, 3FA34789, 0800TS, BOXXBU, BOXXMP, 0101, 5279, 290X2, -108926, N, , 0, 0]]|
|35G71      |[[35G71, 92115301, 08036C, BOXXBU, BOXXMP, 0154, 8380, 35G71, 8003, P, , 0, 0], [35G71, 92115302, 08036C, BOXXBU, BOXXMP, 0144, 8382, 35G71, -2883, N, , 0, 0]]                                                                                  |
+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

    Security: 290X2
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
SECURITY_ID|ACCOUNT_NO |COSTCENTER|    BU|   MPU|LONG_IND|SHORT_IND|QUANTITY|POS_NEG_QUANTITY|PROCESSED|ALLOC_QUANTITY|NET_QUANTITY|
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
290X2      |3FA34789|    0800TS|BOXXBU|BOXXMP|    0101|     5279|      18063|               P|         |             0|           0|
290X2      |3FA34782|    0800TS|BOXXBU|BOXXMP|    0102|     5322|       -863|               N|         |             0|           0|
290X2      |3FA34789|    0800TS|BOXXBU|BOXXMP|    0101|     5279|    -108926|               N|         |             0|           0|
+-------------+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+-------------+

    Security: 35G71
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
SECURITY_ID|ACCOUNT_NO |COSTCENTER|    BU|   MPU|LONG_IND|SHORT_IND|QUANTITY|POS_NEG_QUANTITY|PROCESSED|ALLOC_QUANTITY|NET_QUANTITY|
+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+--------------+------------+
35G71|92115301   |    08036C|BOXXBU|BOXXMP|    0154|     8380      |8003    |               P|         |             0|           0|
35G71|92115302   |    08036C|BOXXBU|BOXXMP|    0144|     8382      |   -2883|               N|         |             0|           0|
+-------------+----------+----------+------+------+--------+---------+-----------+--------+----------------+---------+-------------+

我尝试打开df中的第二个元素，即t（1），并将其转换为数据帧，但失败了。用例类也如下所示：

case class ExplodeWrappedArray(SECURITY_ID: String, ACCOUNT_NO: String, COSTCENTER: String, BU: String, MPU: String, LONG_IND: String, SHORT_IND: String, QUANTITY: String, POS_NEG_QUANTITY: String, PROCESSED: String, ALLOC_QUANTITY: Integer, NET_QUANTITY: Integer)

def printArray(df: DataFrame): Unit = {
    println("Hello")

//This fails
    df.foreach(t => t(1).asInstanceOf[mutable.WrappedArray(ExplodeWrappedArray)])

//This works
    //df.foreach(t => openList((t(1))))
  }

将t（1）传递给openList并直接打印WrappedArray，但我想打开数组并转换为数据帧

def openList(a: mutable.WrappedArray[ExplodeWrappedArray]): Unit = {
    import sparkSession.implicits._
    println("Hello")

    //This works 
    println(a)

    //This fails
    val b = a.toDF("SECURITY_ID", "ACCOUNT_NO", "COSTCENTER", "BU", "MPU", "LONG_IND", "SHORT_IND", "QUANTITY", "POS_NEG_QUANTITY", "PROCESSED", "ALLOC_QUANTITY", "NET_QUANTITY")
    b.printSchema()
    b.show()
  }

---

你能把这个问题简化一点吗？printSchema的输出是什么？printSchema失败，bcoz DF未创建。我想创建DF，这就是我的问题所在。