Sql 基于日期时间列中的间隔时间阈值修改行的id值
我正在研究geolife,它在一个文本文件.plt中包含用户的时间戳GPS跟踪。每个文本文件包含用户一次行程的GPS点。因此,我使用python脚本将数据集导入postgres 由于文件是根据行程的开始时间用数字字符串命名的,因此,例如,下表中包含行程的文件是20070920074804.plt,因此我为行程id会话_id提供了文件名,但不带扩展名。这是这张桌子上的原始GPS轨迹 出于分析目的,我创建了另一个表trips_metrics,其中我从轨迹表计算trip metrics,并将结果插入trip_metrics。我计算的值包括行程距离haversine和持续时间开始时间-结束时间 然后我注意到一些奇怪的事情,一个用户进行了8小时的行程,但行程为321米。彻底浏览行程文件时,我注意到行程时间有跳跃,建议在行程中休息,用户可能停留数小时,然后继续。上表第3行和第4行中有一个示例 为了获得准确的行程时间,我需要对这些情况下的行程进行拆分,如果连续行之间的时间间隔超过30分钟,则应将其视为新的行程,从而获得新的ID 我打算加上数字…02,…03。。在实际计算trips指标(即修改轨迹表)之前,在“我的轨迹表”中输入行程的当前会话id。 因此,对于上表中的示例,我想按如下方式进行拆分:Sql 基于日期时间列中的间隔时间阈值修改行的id值,sql,postgresql,date,window-functions,gaps-and-islands,Sql,Postgresql,Date,Window Functions,Gaps And Islands,我正在研究geolife,它在一个文本文件.plt中包含用户的时间戳GPS跟踪。每个文本文件包含用户一次行程的GPS点。因此,我使用python脚本将数据集导入postgres 由于文件是根据行程的开始时间用数字字符串命名的,因此,例如,下表中包含行程的文件是20070920074804.plt,因此我为行程id会话_id提供了文件名,但不带扩展名。这是这张桌子上的原始GPS轨迹 出于分析目的,我创建了另一个表trips_metrics,其中我从轨迹表计算trip metrics,并将结果插入t
user_id | session_id | timestamp | lat | lon | alt
---------+-------------------+------------------------+-----------+------------+-----
11 | 20070920074804 | 2007-09-20 07:48:04+01 | 28.19737 | 113.006795 | 71
11 | 20070920074804 | 2007-09-20 08:07:09+01 | 28.197685 | 113.006792 | 87
11 | 20070920074804 | 2007-09-20 08:07:10+01 | 28.197685 | 113.00679 | 87
11 | 2007092007480402 | 2007-09-20 14:03:50+01 | 28.197342 | 113.007422 | 62
11 | 2007092007480402 | 2007-09-20 14:04:59+01 | 28.197108 | 113.00734 | 62
11 | 2007092007480402 | 2007-09-20 14:05:01+01 | 28.197088 | 113.00727 | 62
请注意我是如何为新行程分配session_id的,因为两者之间的时间超过30分钟
我如何在postgres中修改或更改我的原始GPS表轨迹
编辑
答:@GMB答案中的第一个查询是有效的,但是,它在new_session_id列中为每一行提供了一个新的session_id
预期结果:
我们的想法是通过旧的会话id+01为新兴的旅行提供一个新的id。如果遇到另一个新的行程,则应为其分配旧的会话id+02,以此类推
B:第二个带更新选项的查询包含语法错误:
update trajectories t
from (
select
t.*,
case when sum(is_gap) over(partition by session_id order by timestamp) > 0
then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
else session_id
end new_session_id
from (
select
t.*,
(timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
from trajectories t
) t
) t1
set session_id = t1.new_session_id
where t1.session_id = t.session_id and t1.timestamp = t.timestamp
ERROR: syntax error at or near "from"
LINE 2: from (
您可以使用lag(一个累计总和)来标识分段,然后使用某种方式来调整会话id:
这是一个空白和孤岛问题。您希望检测时间戳差异大于30分钟的连续行,然后相应地更改会话id 一个选项是使用lag,然后是间隙的累积计数-然后可以使用该信息计算新的会话id: 如果需要,可以将其转换为update语句:
update trajectories t
set session_id = t1.new_session_id
from (
select
t.*,
case when sum(is_gap) over(partition by session_id order by timestamp) > 0
then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
else session_id
end new_session_id
from (
select
t.*,
(timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
from trajectories t
) t
) t1
where t1.session_id = t.session_id and t1.timestamp = t.timestamp
谢谢你的回答。我真的很想用update选项来回答这个问题,但是它需要像我的问题编辑中所说的那样重新阅读。@someone:my bad,Postgres希望set子句在from子句之前。已修复。获取SQL语法错误:错误:关系t不存在第9行:从t@someone . . . 这是您的表名。在这个db提琴中,我说明了错误。看来这个答案考虑了过滤要求,30分钟间隔我不明白为什么postgres总是抱怨子查询别名。你能看看我上面评论中的db提琴吗?@someone。哎呀,那应该是一个窗口函数。我修复了查询并添加了一个dbfiddle。
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
| user_id | session_id | timestamp | lat | lon | alt | is_gap | new_session_id |
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
| 11 | 20070920074804 | 2007-09-20 07:48:04+01 | 28.19737 | 113.006795 | 71 | | 20070920074804 |
| 11 | 20070920074804 | 2007-09-20 08:07:09+01 | 28.197685 | 113.006792 | 87 | | 20070920074804 |
| 11 | 20070920074804 | 2007-09-20 08:07:10+01 | 28.197685 | 113.00679 | 87 | 1 | 2007092007480401 |
| 11 | 20070920074804 | 2007-09-20 14:03:50+01 | 28.197342 | 113.007422 | 62 | 1 | 2007092007480401 |
| 11 | 20070920074804 | 2007-09-20 14:04:59+01 | 28.197108 | 113.00734 | 62 | 1 | 2007092007480401 |
| 11 | 20070920074804 | 2007-09-20 14:05:01+01 | 28.197088 | 113.00727 | 62 | 1 | 2007092007480401 |
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
update trajectories t
from (
select
t.*,
case when sum(is_gap) over(partition by session_id order by timestamp) > 0
then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
else session_id
end new_session_id
from (
select
t.*,
(timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
from trajectories t
) t
) t1
set session_id = t1.new_session_id
where t1.session_id = t.session_id and t1.timestamp = t.timestamp
ERROR: syntax error at or near "from"
LINE 2: from (
select (case when grp >= 1 then session_id * 100 + grp
else session_id
end) as new_session_id,
t.*
from (select t.*,
count(*) filter (where prev_ts < timestamp - interval '30 minute') over (partition by session_id, order by timestamp) as grp
from (select t.*,
lag(timestamp) over (partition by session_id order by timestamp) as prev_ts
from trajectories t
) t
) t;
select
t.*,
case when sum(is_gap) over(partition by session_id order by timestamp) > 0
then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
else session_id
end new_session_id
from (
select
t.*,
(timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
from trajectories t
) t
update trajectories t
set session_id = t1.new_session_id
from (
select
t.*,
case when sum(is_gap) over(partition by session_id order by timestamp) > 0
then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
else session_id
end new_session_id
from (
select
t.*,
(timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
from trajectories t
) t
) t1
where t1.session_id = t.session_id and t1.timestamp = t.timestamp