Pyspark 聚合列值并在新列中更新

我想聚合一列并汇总特定列的值。我想将其添加为其中的一部分 用于进一步计算的相同数据帧

我就是这样做到的

travelGroup = travel.groupBy("day" ).agg(sum("action").cast("int").alias('dayCount'))
travel = travel.join(travelGroup, ['day'],  "left_outer").na.fill(0)

但是，我想检查一下这是否是执行此过程的正确方法或任何其他优化方法

旅行数据帧

---

结果-数据帧

---

您可以调整窗口函数来实现这一点。 在互联网上发现的一个例子是-

case class Salary(depName: String, empNo: Long, salary: Long)
val empsalary = Seq(
  Salary("sales", 1, 5000),
  Salary("personnel", 2, 3900),
  Salary("sales", 3, 4800),
  Salary("sales", 4, 4800),
  Salary("personnel", 5, 3500),
  Salary("develop", 7, 4200),
  Salary("develop", 8, 6000),
  Salary("develop", 9, 4500),
  Salary("develop", 10, 5200),
  Salary("develop", 11, 5200)).toDS

val byDepName = Window.partitionBy('depName)
empsalary.withColumn("avg", avg('salary) over byDepName).show

Pyspark窗口函数-

+---------+---+  +---+  
| day    | action | dayCount
+---------------------
| TUE    | 5      | 13
| WED    | 7      | 7
| TUE    | 2      | 13
| FRI    | 1      | 4
| TUE    | 6      | 13
| FRI    | 3      | 4

case class Salary(depName: String, empNo: Long, salary: Long)
val empsalary = Seq(
  Salary("sales", 1, 5000),
  Salary("personnel", 2, 3900),
  Salary("sales", 3, 4800),
  Salary("sales", 4, 4800),
  Salary("personnel", 5, 3500),
  Salary("develop", 7, 4200),
  Salary("develop", 8, 6000),
  Salary("develop", 9, 4500),
  Salary("develop", 10, 5200),
  Salary("develop", 11, 5200)).toDS

val byDepName = Window.partitionBy('depName)
empsalary.withColumn("avg", avg('salary) over byDepName).show