使用Scala case类为Spark表生成动态标头 我有一个现有的case类，它有许多字段 案例类输出{ userId:String， 时间戳：字符串， ... } 我用它来为这样一个spark工作生成标题。 -------------------- 用户ID |时间戳| -------------------- 1 2324444444 2 2334445556 现在，我想在此添加更多列，这些列将作为attributeName来自映射（attributeName，attributeValue）。所以我的问题是如何将map添加到case类，然后如何使用map key作为列值来生成动态列。在此之后，我的最终输出应该是 ---------------------------------------------------- 用户ID |时间戳| attributeName1 | attributeName2 ---------------------------------------------------- 1 2324444444| | 2 2334445554| |

你可以这样做

I have an existing case class having many fields case class output { userId : String, timeStamp: String, ... } And I am using it to generate header for a spark job like this. -------------------- userId | timeStamp| -------------------- 1 2324444444 2 2334445556 Now i want to add more columns to this and these column will be come from
map(attributeName, attributeValue) as attributeNames. So my question
is how can I add map to case class and then how can i use map key as
column value to generate dynamic columns. After this my final output
should be like ---------------------------------------------------- userId | timeStamp| attributeName1 | attributeName2 ---------------------------------------------------- 1 2324444444| | 2 2334445554| |

---

或者您可以检查其他方法。

检查-->实际上，我想知道我是否可以使用case类来完成此操作。如果是，那么感谢@kavetiraviteja

 case class output {
       userId : String, 
       timeStamp: String,
       keyvalues: Map, 
       ...
    }
    import spark.implicits._
    import org.apache.spark.sql.functions._

    val df = spark.read.textFile(inputlocation).as[output]
    val keysDF = df.select(explode(map_keys($"keyvalues"))).distinct()
    val keys = keysDF.collect().map(f=>f.get(0)).map(f=>col("keyvalues").getItem(f).as(f.toString))
    df.select(col("userId") +: keyCols:_*)