Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/http/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Google analytics BigQuery:计划查询错误:查询执行期间超出了资源:没有足够的资源用于查询计划_Google Analytics_Google Bigquery - Fatal编程技术网

Google analytics BigQuery:计划查询错误:查询执行期间超出了资源:没有足够的资源用于查询计划

Google analytics BigQuery:计划查询错误:查询执行期间超出了资源:没有足够的资源用于查询计划,google-analytics,google-bigquery,Google Analytics,Google Bigquery,在BigQuery中,执行计划查询后出现以下错误: 查询执行期间超出了资源:没有足够的资源用于 查询计划-子查询太多或查询太复杂 我承认这个查询相当复杂,有多个OVER()子句,包括OVER()子句中的PARTITION BY和ORDER BY,从计算角度来看,这是非常昂贵的。然而,这是实现预期结果所必需的。我需要这个OVER()子句来获得所需的结果表。该查询大约为50GB 计划查询查询超过4天的Google Analytics相关数据 然而,值得注意的是,当我手动运行相同的查询时,查询的执行没

在BigQuery中,执行计划查询后出现以下错误:

查询执行期间超出了资源:没有足够的资源用于 查询计划-子查询太多或查询太复杂

我承认这个查询相当复杂,有多个OVER()子句,包括OVER()子句中的PARTITION BY和ORDER BY,从计算角度来看,这是非常昂贵的。然而,这是实现预期结果所必需的。我需要这个OVER()子句来获得所需的结果表。该查询大约为50GB

计划查询查询超过4天的Google Analytics相关数据

然而,值得注意的是,当我手动运行相同的查询时,查询的执行没有任何问题(大约35秒的查询时间)。即使我使用365天的GA数据手动执行查询,查询也会成功执行。此查询为4TB(大约280秒查询时间)

有人知道为什么在我的情况下,计划查询会失败,而手动查询可以无错误地执行吗?考虑到调度非常重要的事实,有人知道是否有一个修复程序可以使调度的查询在没有错误的情况下执行吗

基本上,就是这个查询,见下文。请注意,我隐藏了输入表,因此稍微缩短了查询长度。输入表只是使用UNIONALL合并多个输入表的SELECT查询的集合

还要注意的是,我正在尝试将来自不同来源的点击,Firebase Analytics(应用程序数据)和Universal Analytics(web数据),连接到需要的自定义会话id,如果不需要,请使用GA的常规访问id

SELECT
  *,
  MAX(device) OVER (PARTITION BY country, date, custvisitid_unified RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS device_new,
IF
  (mix_app_web_session = 'mixed',
    CONCAT('mixed_',MAX(app_os) OVER (PARTITION BY country, date, custvisitid_unified RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)),
    browser) AS browser_new,
  MAX(channel) OVER (PARTITION BY country, date, custvisitid_unified RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS channel_new,
IF
  (mix_app_web_session = 'mixed',
    MAX(app_os) OVER (PARTITION BY country, date, custvisitid_unified RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING),
    os) AS os_new
FROM (
  SELECT
    *,
  IF
    (COUNT(DISTINCT webshop) OVER (PARTITION BY country, date, custvisitid_unified) > 1,
      'mixed',
      'single') AS mix_app_web_session
  FROM ( # define whether custvisitid_unified has hits from both app and web
    SELECT
      *,
    IF
      (user_id_anonymous_wide IS NULL
        AND mix_app_web_user = 'single',
        custvisitid,
        CONCAT(MAX(custvisitid) OVER (PARTITION BY country, date, user_id_anonymous_wide RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING),cust_session_counter)) AS custvisitid_unified
    FROM ( # create custvisitid_unified, in which the linked app and web hits have been assigned a unified custom visitid. Only apply a custom visitid to user_id_anonymous_wide = 'mixed' hits (since it is a bit tricky), otherwise just use the regular visitid from GA
      SELECT
        *,
      IF
        (COUNT(DISTINCT webshop) OVER (PARTITION BY country, date, user_id_anonymous_wide) > 1,
          'mixed',
          'single') AS mix_app_web_user,
        (COUNT(new_session) OVER (PARTITION BY country, date, user_id_anonymous_wide ORDER BY timestamp_microsec)) + 1 AS cust_session_counter
      FROM ( # define session counter
        SELECT
          *,
        IF
          ((timestamp_microsec-prev_timestamp_microsec) > 2400000000,
            'new',
            NULL) AS new_session
        FROM ( # Where timestamp is greater than 40 mins (actually 30 mins, but some margin is applied to be sure)
          SELECT
            *,
          IF
            (user_id_anonymous_wide IS NOT NULL,
              LAG(timestamp_microsec,1) OVER (PARTITION BY country, date, user_id_anonymous_wide ORDER BY timestamp_microsec),
              NULL) AS prev_timestamp_microsec
          FROM ( # define previous timestamp to calculate difference in timestamp between consecutive hits
            SELECT
              *,
              MAX(user_id_anonymous) OVER (PARTITION BY country, date, custvisitid RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS user_id_anonymous_wide,
            IF
              (webshop = 'appshop',
                os,
                NULL) AS app_os # user_id_anonymous_wide: define the user_id_anonymous values for all hits within the existing sessionid (initially only in 1 of the hits)
            FROM (

                # SELECT many dimensions FROM multiple tables (which resulted in 1 table with the use of UNION ALL's

                  ) ))))))

更新:我修复了这个问题,将查询分为两个查询。

我建议您使用GROUP by优化查询,然后使用行级聚合,而不是表级聚合。你能在这里添加查询吗?@SabriKaragönen,谢谢你的评论。我在描述中包含了这个查询。请注意,我隐藏了输入表的基本查询,因为这部分相当长,并且与所描述的问题无关(没有应用复杂的计算,只选择了输入字段)。