Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/csharp-4.0/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 如何从数据帧中选择聚合列_Apache Spark_Pyspark_Apache Spark Sql_Pyspark Dataframes - Fatal编程技术网

Apache spark 如何从数据帧中选择聚合列

Apache spark 如何从数据帧中选择聚合列,apache-spark,pyspark,apache-spark-sql,pyspark-dataframes,Apache Spark,Pyspark,Apache Spark Sql,Pyspark Dataframes,df=df.groupby(F.upper(F.col('count'))).agg({'totalvis.age':'avg')).show()创建avg(totalvis.age ASage)列 我想使用另一个聚合函数来选择新创建列的最大值,但存在列名无法解决的问题。您可以使用以下语法为聚合分配列别名: import pyspark.sql.functions as F df2 = df.groupby(F.upper(F.col('county'))).agg(F.avg('totalv

df=df.groupby(F.upper(F.col('count'))).agg({'totalvis.age':'avg')).show()
创建
avg(totalvis.age AS
age


我想使用另一个聚合函数来选择新创建列的最大值,但存在列名无法解决的问题。

您可以使用以下语法为聚合分配列别名:

import pyspark.sql.functions as F

df2 = df.groupby(F.upper(F.col('county'))).agg(F.avg('totalvisitor.age').alias('age_avg'))
然后您可以选择最大值作为
df2。选择(F.max('age\u avg'))

请注意,在您在问题中提供的代码中,您在调用后用
None
覆盖了
df

df = df.(...).show()

因为
df.show()
返回
None

谢谢mck,我该如何选择最大平均年龄的县?@RaviShastri你可以做
df3=df2.filter(F.col('age\u avg')==df2.select(F.max('age\u avg')).head([0])