Sql 雪花服务器，按小时划分持续时间？
我在一个具有以下结构的表中有数据，其中所有Sql 雪花服务器，按小时划分持续时间？,sql,snowflake-cloud-data-platform,Sql,Snowflake Cloud Data Platform,我在一个具有以下结构的表中有数据，其中所有id都是唯一的，并且有许多不同的开始和结束组合，开始和结束之间的差异从不到一分钟到数百分钟不等。我只需要为大于60分钟的开始/结束差异划分持续时间 | ID | DURATION_START | DURATION_END | |--------|-------------------------|-------------------------| | 0abc23 | 2019-06-29 00:08:0
id| ID | DURATION_START | DURATION_END |
|--------|-------------------------|-------------------------|
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 |
| ID | DURATION_START | DURATION_END |
|--------|-------------------------|-------------------------|
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 01:07:59.000 |
| 0abc23 | 2019-06-29 01:08:00.000 | 2019-06-29 02:07:59.000 |
| 0abc23 | 2019-06-29 02:08:00.000 | 2019-06-29 03:07:59.000 |
| 0abc23 | 2019-06-29 03:08:00.000 | 2019-06-29 04:07:59.000 |
| 0abc23 | 2019-06-29 04:08:00.000 | 2019-06-29 05:18:00.000 |
连接方式级别级别select dateadd(hour,24,duration_start)
, greatest(duration_start, date_trunc(hour,dateadd(hour,((level-1)::int/24)::int,duration_start)))
, least(duration_start, date_trunc(hour,dateadd(hour,((level)::int/24)::int,duration_start)))
from test_data
connect by prior LEVEL = floor(datediff(hour, duration_start, duration_end)::int*24)+1;
connectbyselect *
from test_data
connect by prior LEVEL = floor(datediff(hour, duration_start, duration_end)::int*24)+1;
SQL编译错误：错误行0位于位置-1无效标识符“HOUR”如果有人能告诉我如何调整我的雪花方法，那将是非常感谢，谢谢 在纯SQL中实现这一点可以通过各种技巧来实现，但到目前为止，我认为最简单的方法是为此创建一个缩写 下面是一个完整的示例，其中包含一些用于测试的附加行。请注意，可能需要另外涵盖一些特殊情况，例如，如果您的结束时间早于开始时间，或者其中一个为空，该怎么办。但这应该会有所帮助
create or replace table x(
id varchar,
duration_start timestamp_ntz,
duration_end timestamp_ntz
) as
select * from values
('0abc23', '2019-06-29 00:08:00.000', '2019-06-29 09:18:00.000'),
('id_2__', '2002-02-02 02:00:00.000', '2002-02-02 02:00:00.000'),
('id_3__', '2003-03-03 03:00:00.000', '2003-03-03 04:00:00.000'),
('id_4__', '2004-04-04 04:00:00.000', '2004-04-04 04:59:59.000');
create or replace function magic(
id varchar,
duration_start timestamp_ntz,
duration_end timestamp_ntz
)
returns table (
hour_start timestamp_ntz,
hour_end timestamp_ntz
) language javascript as
$$
{
processRow: function(row, rowWriter, context) {
let msSecond = 1000;
let msHour = 60 * 60 * msSecond;
let msStart = row.DURATION_START.getTime();
let msEnd = row.DURATION_END.getTime();
while (msStart <= msEnd) {
let curEnd = Math.min(msEnd, msStart + msHour - msSecond);
rowWriter.writeRow({
HOUR_START: new Date(msStart),
HOUR_END: new Date(curEnd)
});
msStart = curEnd + msSecond;
}
}
}
$$;
select * from x, table(magic(id, duration_start, duration_end)) ;
--------+-------------------------+-------------------------+-------------------------+-------------------------+
ID | DURATION_START | DURATION_END | HOUR_START | HOUR_END |
--------+-------------------------+-------------------------+-------------------------+-------------------------+
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:08:00.000 | 2019-06-29 01:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 01:08:00.000 | 2019-06-29 02:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 02:08:00.000 | 2019-06-29 03:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 03:08:00.000 | 2019-06-29 04:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 04:08:00.000 | 2019-06-29 05:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 05:08:00.000 | 2019-06-29 06:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 06:08:00.000 | 2019-06-29 07:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 07:08:00.000 | 2019-06-29 08:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 08:08:00.000 | 2019-06-29 09:07:59.000 |
0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 09:08:00.000 | 2019-06-29 09:18:00.000 |
id_2__ | 2002-02-02 02:00:00.000 | 2002-02-02 02:00:00.000 | 2002-02-02 02:00:00.000 | 2002-02-02 02:00:00.000 |
id_3__ | 2003-03-03 03:00:00.000 | 2003-03-03 04:00:00.000 | 2003-03-03 03:00:00.000 | 2003-03-03 03:59:59.000 |
id_3__ | 2003-03-03 03:00:00.000 | 2003-03-03 04:00:00.000 | 2003-03-03 04:00:00.000 | 2003-03-03 04:00:00.000 |
id_4__ | 2004-04-04 04:00:00.000 | 2004-04-04 04:59:59.000 | 2004-04-04 04:00:00.000 | 2004-04-04 04:59:59.000 |
--------+-------------------------+-------------------------+-------------------------+-------------------------+
创建或替换表x(
id varchar，
持续时间\u开始时间戳\u ntz，
持续时间\u结束时间戳\u ntz
)作为
从值中选择*
（'0abc23'、'2019-06-29 00:08:00.000'、'2019-06-29 09:18:00.000'），
（'id_2__'，'2002-02-02 02:00:00.000'，'2002-02-02 02:00:00.000'），
（'id_3_'、'2003-03-03 03:00:00.000'、'2003-03-03 04:00:00.000'），
（'id_4__'，'2004-04-04 04:00:00.000'，'2004-04-04 04:59:59.000'）；
创建或替换函数魔术(
id varchar，
持续时间\u开始时间戳\u ntz，
持续时间\u结束时间戳\u ntz
)
返回表(
小时\u开始时间戳\u ntz，
小时\u结束时间戳\u ntz
)语言javascript作为
$$
{
processRow：函数（行、行编写器、上下文）{
设msSecond=1000；
设msHour=60*60*ms秒；
让msStart=row.DURATION\u START.getTime（）；
让msEnd=row.DURATION_END.getTime（）；
而（msStart对于这种类型的项目，有单独的日期和间隔表是有帮助的
下面是实现所需功能的脚本
select
f.id
,f.DURATION_START
,f.DURATION_END
,f.start_time_HOUR_START
,f.end_time_HOUR_START
,q.CALENDAR_DATE
,q.HOUR_START
,q.HOUR_END
,case
-- when starts during interval and ends after interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, dateadd(hour, 1, q.HOUR_START))
-- when starts during interval and ends during interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, f.DURATION_END)
-- when starts before interval and ends during interval
when f.DURATION_START <= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, f.DURATION_END)
-- entire interval , starts before, and ends after
when
f.DURATION_START <= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, dateadd(hour, 1, q.HOUR_START))
else 0 end as seconds_elapsed
from (
select *
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_START) - (datediff(s, '1970-01-01', DURATION_START) % 3600),
'1970-01-01')) as start_time_HOUR_START
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_END) - (datediff(s, '1970-01-01', DURATION_END) % 3600),
'1970-01-01')) as end_time_HOUR_START
from example1
) f
inner join
(
select
distinct
q1.calendar_date
-- , t2.rid2
, dateadd(hour, t2.id, to_timestamp(q1.calendar_date)) as HOUR_START
, dateadd(hour, t2.id + 1, to_timestamp(q1.calendar_date)) as HOUR_END
from (
select calendar_date
from calendar
where calendar_date between (select to_date(min(DURATION_START)) from example1) and (select to_date(max(DURATION_END)) from example1)
) q1
cross join
interval as t2
-- order by HOUR_START
) q on q.HOUR_START between f.start_time_HOUR_START and f.end_time_HOUR_START
ORDER BY f.id
, f.DURATION_START
, f.DURATION_END
, q.CALENDAR_DATE
, q.HOUR_START
;
创建具有所需日期范围的日历表
select
f.id
,f.DURATION_START
,f.DURATION_END
,f.start_time_HOUR_START
,f.end_time_HOUR_START
,q.CALENDAR_DATE
,q.HOUR_START
,q.HOUR_END
,case
-- when starts during interval and ends after interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, dateadd(hour, 1, q.HOUR_START))
-- when starts during interval and ends during interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, f.DURATION_END)
-- when starts before interval and ends during interval
when f.DURATION_START <= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, f.DURATION_END)
-- entire interval , starts before, and ends after
when
f.DURATION_START <= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, dateadd(hour, 1, q.HOUR_START))
else 0 end as seconds_elapsed
from (
select *
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_START) - (datediff(s, '1970-01-01', DURATION_START) % 3600),
'1970-01-01')) as start_time_HOUR_START
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_END) - (datediff(s, '1970-01-01', DURATION_END) % 3600),
'1970-01-01')) as end_time_HOUR_START
from example1
) f
inner join
(
select
distinct
q1.calendar_date
-- , t2.rid2
, dateadd(hour, t2.id, to_timestamp(q1.calendar_date)) as HOUR_START
, dateadd(hour, t2.id + 1, to_timestamp(q1.calendar_date)) as HOUR_END
from (
select calendar_date
from calendar
where calendar_date between (select to_date(min(DURATION_START)) from example1) and (select to_date(max(DURATION_END)) from example1)
) q1
cross join
interval as t2
-- order by HOUR_START
) q on q.HOUR_START between f.start_time_HOUR_START and f.end_time_HOUR_START
ORDER BY f.id
, f.DURATION_START
, f.DURATION_END
, q.CALENDAR_DATE
, q.HOUR_START
;
我从unix时代的“1970-01-01”开始，因为在这种情况下，我会随时准备一个日历表
create or replace table calendar(calendar_date DATE)
insert into calendar(calendar_date)
select
dateadd(d,rid2,to_timestamp_ntz('1970-01-01')) as calendar_date
from
(
select 0 as rid2 union
select row_number() over (order by null) as rid2
from table (generator(rowcount => 100000))
) t
where dateadd(d,rid2,to_timestamp_ntz('1970-01-01')) < '2030-01-01'
order by 1 asc;
接下来，我创建了一个包含示例数据以及其他几个值的表，以便可以跨不同场景验证计算
create or replace table example1(id varchar(10), DURATION_START datetime, DURATION_END datetime);
-- drop table example1
truncate table example1;
--
insert into example1 values('0abc23','2019-06-29 00:08:00.000','2019-06-29 09:18:00.000');
insert into example1 values('0abc24','2019-06-28 11:07:45.000','2019-06-28 12:08:45.000');
insert into example1 values('0abc25','2019-06-28 01:00:00.000','2019-06-29 02:15:00.000');
insert into example1 values('0abc26','2019-06-28 00:08:00.000','2019-06-29 15:18:00.000');
假设一切都已设置好，下面的查询将为您提供所需的结果
select
f.id
,f.DURATION_START
,f.DURATION_END
,f.start_time_HOUR_START
,f.end_time_HOUR_START
,q.CALENDAR_DATE
,q.HOUR_START
,q.HOUR_END
,case
-- when starts during interval and ends after interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, dateadd(hour, 1, q.HOUR_START))
-- when starts during interval and ends during interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, f.DURATION_END)
-- when starts before interval and ends during interval
when f.DURATION_START <= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, f.DURATION_END)
-- entire interval , starts before, and ends after
when
f.DURATION_START <= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, dateadd(hour, 1, q.HOUR_START))
else 0 end as seconds_elapsed
from (
select *
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_START) - (datediff(s, '1970-01-01', DURATION_START) % 3600),
'1970-01-01')) as start_time_HOUR_START
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_END) - (datediff(s, '1970-01-01', DURATION_END) % 3600),
'1970-01-01')) as end_time_HOUR_START
from example1
) f
inner join
(
select
distinct
q1.calendar_date
-- , t2.rid2
, dateadd(hour, t2.id, to_timestamp(q1.calendar_date)) as HOUR_START
, dateadd(hour, t2.id + 1, to_timestamp(q1.calendar_date)) as HOUR_END
from (
select calendar_date
from calendar
where calendar_date between (select to_date(min(DURATION_START)) from example1) and (select to_date(max(DURATION_END)) from example1)
) q1
cross join
interval as t2
-- order by HOUR_START
) q on q.HOUR_START between f.start_time_HOUR_START and f.end_time_HOUR_START
ORDER BY f.id
, f.DURATION_START
, f.DURATION_END
, q.CALENDAR_DATE
, q.HOUR_START
;
对于代码链接使用JS UDTF与sql相比，效率如何？这取决于。对于简单的操作，它会更慢。但是如果sql中的逻辑很难或很复杂，它通常会更快（并且更容易实现）。在这种情况下，因为我不确定如何在纯sql中实现它，所以很难确定。
select
f.id
,f.DURATION_START
,f.DURATION_END
,f.start_time_HOUR_START
,f.end_time_HOUR_START
,q.CALENDAR_DATE
,q.HOUR_START
,q.HOUR_END
,case
-- when starts during interval and ends after interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, dateadd(hour, 1, q.HOUR_START))
-- when starts during interval and ends during interval
when f.DURATION_START >= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, f.DURATION_START, f.DURATION_END)
-- when starts before interval and ends during interval
when f.DURATION_START <= q.HOUR_START and f.DURATION_END <= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, f.DURATION_END)
-- entire interval , starts before, and ends after
when
f.DURATION_START <= q.HOUR_START and f.DURATION_END >= dateadd(hour, 1, q.HOUR_START)
then datediff(s, q.HOUR_START, dateadd(hour, 1, q.HOUR_START))
else 0 end as seconds_elapsed
from (
select *
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_START) - (datediff(s, '1970-01-01', DURATION_START) % 3600),
'1970-01-01')) as start_time_HOUR_START
, to_timestamp(
dateadd(s, datediff(s, '1970-01-01', DURATION_END) - (datediff(s, '1970-01-01', DURATION_END) % 3600),
'1970-01-01')) as end_time_HOUR_START
from example1
) f
inner join
(
select
distinct
q1.calendar_date
-- , t2.rid2
, dateadd(hour, t2.id, to_timestamp(q1.calendar_date)) as HOUR_START
, dateadd(hour, t2.id + 1, to_timestamp(q1.calendar_date)) as HOUR_END
from (
select calendar_date
from calendar
where calendar_date between (select to_date(min(DURATION_START)) from example1) and (select to_date(max(DURATION_END)) from example1)
) q1
cross join
interval as t2
-- order by HOUR_START
) q on q.HOUR_START between f.start_time_HOUR_START and f.end_time_HOUR_START
ORDER BY f.id
, f.DURATION_START
, f.DURATION_END
, q.CALENDAR_DATE
, q.HOUR_START
;
+--------+-------------------------+-------------------------+-------------------------+-------------------------+---------------+-------------------------+-------------------------+-----------------+
| ID | DURATION_START | DURATION_END | START_TIME_HOUR_START | END_TIME_HOUR_START | CALENDAR_DATE | HOUR_START | HOUR_END | SECONDS_ELAPSED |
|--------+-------------------------+-------------------------+-------------------------+-------------------------+---------------+-------------------------+-------------------------+-----------------|
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 00:00:00.000 | 2019-06-29 01:00:00.000 | 3120 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 01:00:00.000 | 2019-06-29 02:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 02:00:00.000 | 2019-06-29 03:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 03:00:00.000 | 2019-06-29 04:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 04:00:00.000 | 2019-06-29 05:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 05:00:00.000 | 2019-06-29 06:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 06:00:00.000 | 2019-06-29 07:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 07:00:00.000 | 2019-06-29 08:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 08:00:00.000 | 2019-06-29 09:00:00.000 | 3600 |
| 0abc23 | 2019-06-29 00:08:00.000 | 2019-06-29 09:18:00.000 | 2019-06-29 00:00:00.000 | 2019-06-29 09:00:00.000 | 2019-06-29 | 2019-06-29 09:00:00.000 | 2019-06-29 10:00:00.000 | 1080 |
| 0abc24 | 2019-06-28 11:07:45.000 | 2019-06-28 12:08:45.000 | 2019-06-28 11:00:00.000 | 2019-06-28 12:00:00.000 | 2019-06-28 | 2019-06-28 11:00:00.000 | 2019-06-28 12:00:00.000 | 3135 |
| 0abc24 | 2019-06-28 11:07:45.000 | 2019-06-28 12:08:45.000 | 2019-06-28 11:00:00.000 | 2019-06-28 12:00:00.000 | 2019-06-28 | 2019-06-28 12:00:00.000 | 2019-06-28 13:00:00.000 | 525 |