日期时间聚合组的性能Postgresql 9.4

日期时间聚合组的性能Postgresql 9.4,postgresql,query-performance,Postgresql,Query Performance,需求是计算一组相互叠加的聚合,最终得到JSON。我遵循两条准则:先使用group by,然后使用count,然后使用SELECT(DISTINCT),并且我尝试减少每个连续CTE涉及的行数。 我最终得到的是一个相当冗长的查询,没有那么快。 所以我的问题是:在这种情况下,值得研究递归CTE吗?或者我应该研究的任何其他技巧/方向,以减少查询大小和/或提高性能 准备好的声明 分析输出: QUERY PLAN Result (cost=8873.92..8873.94 rows=1 width=0)

需求是计算一组相互叠加的聚合,最终得到JSON。我遵循两条准则:先使用group by,然后使用count,然后使用SELECT(DISTINCT),并且我尝试减少每个连续CTE涉及的行数。 我最终得到的是一个相当冗长的查询,没有那么快。 所以我的问题是:在这种情况下,值得研究递归CTE吗?或者我应该研究的任何其他技巧/方向,以减少查询大小和/或提高性能

准备好的声明

分析输出:

QUERY PLAN
Result  (cost=8873.92..8873.94 rows=1 width=0) (actual time=713.709..713.710 rows=1 loops=1)
  CTE limited_readings_hours
    ->  HashAggregate  (cost=8790.07..8791.50 rows=115 width=42) (actual time=642.622..646.458 rows=4432 loops=1)
          Group Key: date_trunc('hour'::text, rr.issued), rr.sensor_id, rr.value
          ->  Seq Scan on raw_readings rr  (cost=0.00..8783.41 rows=888 width=42) (actual time=0.034..441.787 rows=177545 loops=1)
                Filter: ((sensor_id = ANY ('{54993ab8-c0e8-4359-b7c2-77f00a00011a}'::uuid[])) AND (date_trunc('day'::text, issued) >= date_trunc('day'::text, ('2015-01-01'::date)::timestamp with time zone)) AND (date_trunc('day'::text, issued) <= date_trunc('day'::text, ('2015-01-30'::date)::timestamp with time zone)))
                Rows Removed by Filter: 90
  CTE hours
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=29.800..37.062 rows=200 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_hours.issued_hour), 'IYYY-MM-DDTHH24Z'::text)), limited_readings_hours.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=29.751..33.013 rows=4432 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_hours.issued_hour), 'IYYY-MM-DDTHH24Z'::text)), limited_readings_hours.sensor_id
                Sort Method: quicksort  Memory: 539kB
                ->  CTE Scan on limited_readings_hours  (cost=0.00..2.88 rows=115 width=24) (actual time=0.015..8.542 rows=4432 loops=1)
  CTE limited_readings_days
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=660.996..661.571 rows=697 loops=1)
          Group Key: date_trunc('days'::text, limited_readings_hours_1.issued_hour), limited_readings_hours_1.sensor_id, limited_readings_hours_1.value
          ->  CTE Scan on limited_readings_hours limited_readings_hours_1  (cost=0.00..2.59 rows=115 width=56) (actual time=642.632..656.110 rows=4432 loops=1)
  CTE days
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=3.260..4.254 rows=9 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_days.issued_day), 'IYYY-MM-DD'::text)), limited_readings_days.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=3.133..3.652 rows=697 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_days.issued_day), 'IYYY-MM-DD'::text)), limited_readings_days.sensor_id
                Sort Method: quicksort  Memory: 79kB
                ->  CTE Scan on limited_readings_days  (cost=0.00..2.88 rows=115 width=24) (actual time=0.014..1.254 rows=697 loops=1)
  CTE limited_readings_weeks
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=663.871..664.138 rows=336 loops=1)
          Group Key: date_trunc('weeks'::text, limited_readings_days_1.issued_day), limited_readings_days_1.sensor_id, limited_readings_days_1.value
          ->  CTE Scan on limited_readings_days limited_readings_days_1  (cost=0.00..2.59 rows=115 width=56) (actual time=661.001..663.115 rows=697 loops=1)
  CTE weeks
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=1.545..1.795 rows=2 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_weeks.issued_week), 'IYYY-WIW'::text)), limited_readings_weeks.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=1.246..1.491 rows=336 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_weeks.issued_week), 'IYYY-WIW'::text)), limited_readings_weeks.sensor_id
                Sort Method: quicksort  Memory: 51kB
                ->  CTE Scan on limited_readings_weeks  (cost=0.00..2.88 rows=115 width=24) (actual time=0.012..0.574 rows=336 loops=1)
  CTE limited_readings_months
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=665.277..665.481 rows=248 loops=1)
          Group Key: date_trunc('months'::text, limited_readings_weeks_1.issued_week), limited_readings_weeks_1.sensor_id, limited_readings_weeks_1.value
          ->  CTE Scan on limited_readings_weeks limited_readings_weeks_1  (cost=0.00..2.59 rows=115 width=56) (actual time=663.877..664.885 rows=336 loops=1)
  CTE months
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=1.282..1.283 rows=1 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_months.issued_month), 'IYYY-MM'::text)), limited_readings_months.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=0.866..1.048 rows=248 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_months.issued_month), 'IYYY-MM'::text)), limited_readings_months.sensor_id
                Sort Method: quicksort  Memory: 44kB
                ->  CTE Scan on limited_readings_months  (cost=0.00..2.88 rows=115 width=24) (actual time=0.013..0.430 rows=248 loops=1)
  CTE limited_readings_years
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=666.298..666.491 rows=248 loops=1)
          Group Key: date_trunc('years'::text, limited_readings_months_1.issued_month), limited_readings_months_1.sensor_id, limited_readings_months_1.value
          ->  CTE Scan on limited_readings_months limited_readings_months_1  (cost=0.00..2.59 rows=115 width=56) (actual time=665.287..666.013 rows=248 loops=1)
  CTE years
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=667.923..667.924 rows=1 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_years.issued_year), 'IYYY'::text)), limited_readings_years.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=667.519..667.701 rows=248 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_years.issued_year), 'IYYY'::text)), limited_readings_years.sensor_id
                Sort Method: quicksort  Memory: 44kB
                ->  CTE Scan on limited_readings_years  (cost=0.00..2.88 rows=115 width=24) (actual time=666.315..667.140 rows=248 loops=1)
  InitPlan 11 (returns $10)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=667.986..667.987 rows=1 loops=1)
          ->  CTE Scan on years  (cost=0.00..2.30 rows=115 width=24) (actual time=667.942..667.944 rows=1 loops=1)
  InitPlan 12 (returns $11)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=1.325..1.326 rows=1 loops=1)
          ->  CTE Scan on months  (cost=0.00..2.30 rows=115 width=24) (actual time=1.296..1.298 rows=1 loops=1)
  InitPlan 13 (returns $12)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=1.849..1.850 rows=1 loops=1)
          ->  CTE Scan on weeks  (cost=0.00..2.30 rows=115 width=24) (actual time=1.555..1.809 rows=2 loops=1)
  InitPlan 14 (returns $13)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=4.340..4.341 rows=1 loops=1)
          ->  CTE Scan on days  (cost=0.00..2.30 rows=115 width=24) (actual time=3.274..4.288 rows=9 loops=1)
  InitPlan 15 (returns $14)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=38.143..38.144 rows=1 loops=1)
          ->  CTE Scan on hours  (cost=0.00..2.30 rows=115 width=24) (actual time=29.805..37.483 rows=200 loops=1)
Execution time: 713.993 ms
查询计划
结果(成本=8873.92..8873.94行=1宽度=0)(实际时间=713.709..713.710行=1循环=1)
CTE有限公司
->HashAggregate(成本=8790.07..8791.50行=115宽度=42)(实际时间=642.622..646.458行=4432循环=1)
组键:日期(小时):文本,rr.ISTED),rr.sensor\U id,rr.value
->原始读数rr上的顺序扫描(成本=0.00..8783.41行=888宽度=42)(实际时间=0.034..441.787行=177545循环=1)
过滤器:((传感器id=ANY(“{54993ab8-c0e8-4359-b7c2-77f00a00011a}”):uuid[])和(日期(日期::文本,发布)>=日期(日期:('2015-01-01':日期::带时区的时间戳))和(日期:'date('day':文本,发布)组聚合(成本=6.81..9.69行宽度=115行宽度=24)(实际时间=29.800..37.062行=200圈)
组键:(to_char(时区('UTC'::文本,有限读取时间。发布时间),'IYY-MM-DDTHH24Z'::文本)),有限读取时间。传感器id
->排序(成本=6.81..7.10行=115宽度=24)(实际时间=29.751..33.013行=4432循环=1)
排序键:(to_char(时区('UTC'::文本,有限读取时间。发布时间),'iyy-MM-DDTHH24Z'::文本)),有限读取时间。传感器id
排序方法:快速排序内存:539kB
->有限时间内CTE扫描读数(成本=0.00..2.88行=115宽度=24)(实际时间=0.015..8.542行=4432圈=1)
CTE有限公司
->HashAggregate(成本=3.45..4.89行=115宽度=56)(实际时间=660.996..661.571行=697循环=1)
组键:日期(天):文本,有限读数(小时数)1.发布(小时数),有限读数(小时数)1.传感器id,有限读数(小时数)1.值
->有限读取时间有限读取时间有限(成本=0.00..2.59行=115宽度=56)(实际时间=642.632..656.110行=4432循环=1)
CTE天数
->GroupAggregate(成本=6.81..9.69行=115宽度=24)(实际时间=3.260..4.254行=9圈=1)
组键:(至字符(时区('UTC':文本,有限读数天数。发布日期),'IYYY-MM-DD':文本)),有限读数天数。传感器id
->排序(成本=6.81..7.10行=115宽度=24)(实际时间=3.133..3.652行=697圈=1)
排序键:(to_char(时区('UTC'::文本,有限读数天数。发布日期),'IYYY-MM-DD'::文本)),有限读数天数。传感器id
排序方法:快速排序内存:79kB
->有限天内的CTE扫描(成本=0.00..2.88行=115宽度=24)(实际时间=0.014..1.254行=697圈=1)
CTE有限公司
->HashAggregate(成本=3.45..4.89行=115宽度=56)(实际时间=663.871..664.138行=336个循环=1)
组键:日期(“周”):文本,有限读数,天数,发布日期),有限读数,天数,传感器id,有限读数,天数,值
->有限读数的CTE扫描天数有限读数天数1(成本=0.00..2.59行=115宽度=56)(实际时间=661.001..663.115行=697圈=1)
CTE周
->GroupAggregate(成本=6.81..9.69行=115宽度=24)(实际时间=1.545..1.795行=2圈=1)
组键:(至字符(时区('UTC':文本,有限读数周。发布周),'IYYY-WIW':文本)),有限读数周。传感器id
->排序(成本=6.81..7.10行=115宽度=24)(实际时间=1.246..1.491行=336个循环=1)
排序键:(to_char(时区('UTC'::文本,有限读数周。发布周),'IYYY-WIW'::文本)),有限读数周。传感器id
排序方法:快速排序内存:51kB
->有限读数上的CTE扫描(成本=0.00..2.88行=115宽度=24)(实际时间=0.012..0.574行=336圈=1)
CTE有限公司\u读数\u个月
->HashAggregate(成本=3.45..4.89行=115宽度=56)(实际时间=665.277..665.481行=248循环=1)
组键:日期(“月”):文本,有限读数,周数,发布周数),有限读数,周数,传感器id,有限读数,周数,值
->CTE扫描有限周读数有限周读数1(成本=0.00..2.59行=115宽度=56)(实际时间=663.877..664.885行=336个循环=1)
CTE月份
->GroupAggregate(成本=6.81..9.69行=115宽度=24)(实际时间=1.282..1.283行=1圈=1)
组键:(至字符(时区('UTC':文本,有限读数\u月。发布\u月),'IYYY-MM':文本)),有限读数\u月。传感器id
->排序(成本=6.81..7.10行=115宽度=24)(实际时间=0.866..1.048行=248循环=1)
排序键:(至字符(时区('UTC'::文本,有限读数\u月。发布\u月),'IYYY-MM'::文本)),有限读数\u月。传感器id
排序方法:快速排序内存:44kB
->有限读数上的CTE扫描(成本=0.00..2.88行=115宽度=24)(实际时间=0.013..0.430行=248圈=1)
CTE有限公司年
->HashAggregate(成本=3.45..4.89行=115宽度=56)(实际时间=666.298..666.491行=248循环=1)
组键:日期(“年”):文本、有限读数、月份、发布月份)、有限读数、月份、传感器id、有限读数、月份、值
->有限读数的CTE扫描\u月有限读数\u月\u 1(成本=0.00..2.59行=115宽度=56)(实际时间=665.287..666.013行=248圈=1)
CTE年
->GroupAggregate(成本=6.81..9.69行=115宽度=24)(实际时间=667.923。
CREATE TABLE raw_readings
(
  raw_reading_id uuid NOT NULL,
  created timestamp with time zone DEFAULT now(),
  modified timestamp with time zone,
  sensor_id uuid NOT NULL,
  value text,
  issued timestamp with time zone NOT NULL,
  type text,
  account_id uuid,
  CONSTRAINT raw_readings_pkey PRIMARY KEY (raw_reading_id)
)
QUERY PLAN
Result  (cost=8873.92..8873.94 rows=1 width=0) (actual time=713.709..713.710 rows=1 loops=1)
  CTE limited_readings_hours
    ->  HashAggregate  (cost=8790.07..8791.50 rows=115 width=42) (actual time=642.622..646.458 rows=4432 loops=1)
          Group Key: date_trunc('hour'::text, rr.issued), rr.sensor_id, rr.value
          ->  Seq Scan on raw_readings rr  (cost=0.00..8783.41 rows=888 width=42) (actual time=0.034..441.787 rows=177545 loops=1)
                Filter: ((sensor_id = ANY ('{54993ab8-c0e8-4359-b7c2-77f00a00011a}'::uuid[])) AND (date_trunc('day'::text, issued) >= date_trunc('day'::text, ('2015-01-01'::date)::timestamp with time zone)) AND (date_trunc('day'::text, issued) <= date_trunc('day'::text, ('2015-01-30'::date)::timestamp with time zone)))
                Rows Removed by Filter: 90
  CTE hours
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=29.800..37.062 rows=200 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_hours.issued_hour), 'IYYY-MM-DDTHH24Z'::text)), limited_readings_hours.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=29.751..33.013 rows=4432 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_hours.issued_hour), 'IYYY-MM-DDTHH24Z'::text)), limited_readings_hours.sensor_id
                Sort Method: quicksort  Memory: 539kB
                ->  CTE Scan on limited_readings_hours  (cost=0.00..2.88 rows=115 width=24) (actual time=0.015..8.542 rows=4432 loops=1)
  CTE limited_readings_days
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=660.996..661.571 rows=697 loops=1)
          Group Key: date_trunc('days'::text, limited_readings_hours_1.issued_hour), limited_readings_hours_1.sensor_id, limited_readings_hours_1.value
          ->  CTE Scan on limited_readings_hours limited_readings_hours_1  (cost=0.00..2.59 rows=115 width=56) (actual time=642.632..656.110 rows=4432 loops=1)
  CTE days
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=3.260..4.254 rows=9 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_days.issued_day), 'IYYY-MM-DD'::text)), limited_readings_days.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=3.133..3.652 rows=697 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_days.issued_day), 'IYYY-MM-DD'::text)), limited_readings_days.sensor_id
                Sort Method: quicksort  Memory: 79kB
                ->  CTE Scan on limited_readings_days  (cost=0.00..2.88 rows=115 width=24) (actual time=0.014..1.254 rows=697 loops=1)
  CTE limited_readings_weeks
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=663.871..664.138 rows=336 loops=1)
          Group Key: date_trunc('weeks'::text, limited_readings_days_1.issued_day), limited_readings_days_1.sensor_id, limited_readings_days_1.value
          ->  CTE Scan on limited_readings_days limited_readings_days_1  (cost=0.00..2.59 rows=115 width=56) (actual time=661.001..663.115 rows=697 loops=1)
  CTE weeks
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=1.545..1.795 rows=2 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_weeks.issued_week), 'IYYY-WIW'::text)), limited_readings_weeks.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=1.246..1.491 rows=336 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_weeks.issued_week), 'IYYY-WIW'::text)), limited_readings_weeks.sensor_id
                Sort Method: quicksort  Memory: 51kB
                ->  CTE Scan on limited_readings_weeks  (cost=0.00..2.88 rows=115 width=24) (actual time=0.012..0.574 rows=336 loops=1)
  CTE limited_readings_months
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=665.277..665.481 rows=248 loops=1)
          Group Key: date_trunc('months'::text, limited_readings_weeks_1.issued_week), limited_readings_weeks_1.sensor_id, limited_readings_weeks_1.value
          ->  CTE Scan on limited_readings_weeks limited_readings_weeks_1  (cost=0.00..2.59 rows=115 width=56) (actual time=663.877..664.885 rows=336 loops=1)
  CTE months
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=1.282..1.283 rows=1 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_months.issued_month), 'IYYY-MM'::text)), limited_readings_months.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=0.866..1.048 rows=248 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_months.issued_month), 'IYYY-MM'::text)), limited_readings_months.sensor_id
                Sort Method: quicksort  Memory: 44kB
                ->  CTE Scan on limited_readings_months  (cost=0.00..2.88 rows=115 width=24) (actual time=0.013..0.430 rows=248 loops=1)
  CTE limited_readings_years
    ->  HashAggregate  (cost=3.45..4.89 rows=115 width=56) (actual time=666.298..666.491 rows=248 loops=1)
          Group Key: date_trunc('years'::text, limited_readings_months_1.issued_month), limited_readings_months_1.sensor_id, limited_readings_months_1.value
          ->  CTE Scan on limited_readings_months limited_readings_months_1  (cost=0.00..2.59 rows=115 width=56) (actual time=665.287..666.013 rows=248 loops=1)
  CTE years
    ->  GroupAggregate  (cost=6.81..9.69 rows=115 width=24) (actual time=667.923..667.924 rows=1 loops=1)
          Group Key: (to_char(timezone('UTC'::text, limited_readings_years.issued_year), 'IYYY'::text)), limited_readings_years.sensor_id
          ->  Sort  (cost=6.81..7.10 rows=115 width=24) (actual time=667.519..667.701 rows=248 loops=1)
                Sort Key: (to_char(timezone('UTC'::text, limited_readings_years.issued_year), 'IYYY'::text)), limited_readings_years.sensor_id
                Sort Method: quicksort  Memory: 44kB
                ->  CTE Scan on limited_readings_years  (cost=0.00..2.88 rows=115 width=24) (actual time=666.315..667.140 rows=248 loops=1)
  InitPlan 11 (returns $10)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=667.986..667.987 rows=1 loops=1)
          ->  CTE Scan on years  (cost=0.00..2.30 rows=115 width=24) (actual time=667.942..667.944 rows=1 loops=1)
  InitPlan 12 (returns $11)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=1.325..1.326 rows=1 loops=1)
          ->  CTE Scan on months  (cost=0.00..2.30 rows=115 width=24) (actual time=1.296..1.298 rows=1 loops=1)
  InitPlan 13 (returns $12)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=1.849..1.850 rows=1 loops=1)
          ->  CTE Scan on weeks  (cost=0.00..2.30 rows=115 width=24) (actual time=1.555..1.809 rows=2 loops=1)
  InitPlan 14 (returns $13)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=4.340..4.341 rows=1 loops=1)
          ->  CTE Scan on days  (cost=0.00..2.30 rows=115 width=24) (actual time=3.274..4.288 rows=9 loops=1)
  InitPlan 15 (returns $14)
    ->  Aggregate  (cost=2.88..2.89 rows=1 width=24) (actual time=38.143..38.144 rows=1 loops=1)
          ->  CTE Scan on hours  (cost=0.00..2.30 rows=115 width=24) (actual time=29.805..37.483 rows=200 loops=1)
Execution time: 713.993 ms