Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/sql-server-2008/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark SPARK-如何在分组查询中使用函数_Apache Spark_Shark Sql - Fatal编程技术网

Apache spark SPARK-如何在分组查询中使用函数

Apache spark SPARK-如何在分组查询中使用函数,apache-spark,shark-sql,Apache Spark,Shark Sql,我将把SHARK查询迁移到SPARK中 下面是我的示例SHARK查询,它使用GROUPBY子句中的函数 select month(dt_cr) as Month, day(dt_cr) as date_of_created, count(distinct phone_number) as total_customers from customer group by month(dt_cr),day(dt_cr); 这个查询在SPARK sql中不起作用,它给出了

我将把SHARK查询迁移到SPARK中

下面是我的示例SHARK查询,它使用GROUPBY子句中的函数

select month(dt_cr) as Month,
   day(dt_cr)   as date_of_created,
   count(distinct phone_number) as total_customers        
from customer
group by month(dt_cr),day(dt_cr);
这个查询在SPARK sql中不起作用,它给出了以下错误

错误:org.apache.spark.sql.catalyst.errors.package$TreeNodeException:表达式不在GROUP BY中

所以,作为我下面使用的SPARK查询解决方案的一部分,这是可行的,但需要更改代码。这对我现有的项目影响很大。因此,任何人都有一个影响最小的更好的解决方案

SELECT Month,date_of_created,count(distinct phone_number) as total_customers        
FROM
(select month(dt_cr) as Month,
    day(dt_cr)   as date_of_created,
    email
from customers)A
group by Month,date_of_created

这是Spark SQL中的一个问题:

但是,我认为它将在下一版本中修复。现在,您必须更改代码以绕过此问题