Sql 基于日期时间列中的间隔时间阈值修改行的id值

Sql 基于日期时间列中的间隔时间阈值修改行的id值,sql,postgresql,date,window-functions,gaps-and-islands,Sql,Postgresql,Date,Window Functions,Gaps And Islands,我正在研究geolife,它在一个文本文件.plt中包含用户的时间戳GPS跟踪。每个文本文件包含用户一次行程的GPS点。因此,我使用python脚本将数据集导入postgres 由于文件是根据行程的开始时间用数字字符串命名的,因此,例如,下表中包含行程的文件是20070920074804.plt,因此我为行程id会话_id提供了文件名,但不带扩展名。这是这张桌子上的原始GPS轨迹 出于分析目的,我创建了另一个表trips_metrics,其中我从轨迹表计算trip metrics,并将结果插入t

我正在研究geolife,它在一个文本文件.plt中包含用户的时间戳GPS跟踪。每个文本文件包含用户一次行程的GPS点。因此,我使用python脚本将数据集导入postgres

由于文件是根据行程的开始时间用数字字符串命名的,因此,例如,下表中包含行程的文件是20070920074804.plt,因此我为行程id会话_id提供了文件名,但不带扩展名。这是这张桌子上的原始GPS轨迹

出于分析目的,我创建了另一个表trips_metrics,其中我从轨迹表计算trip metrics,并将结果插入trip_metrics。我计算的值包括行程距离haversine和持续时间开始时间-结束时间

然后我注意到一些奇怪的事情,一个用户进行了8小时的行程,但行程为321米。彻底浏览行程文件时,我注意到行程时间有跳跃,建议在行程中休息,用户可能停留数小时,然后继续。上表第3行和第4行中有一个示例

为了获得准确的行程时间,我需要对这些情况下的行程进行拆分,如果连续行之间的时间间隔超过30分钟,则应将其视为新的行程,从而获得新的ID

我打算加上数字…02,…03。。在实际计算trips指标(即修改轨迹表)之前,在“我的轨迹表”中输入行程的当前会话id。 因此,对于上表中的示例,我想按如下方式进行拆分:

 user_id |    session_id     |       timestamp        |    lat    |    lon     | alt 
---------+-------------------+------------------------+-----------+------------+-----
      11 |  20070920074804   | 2007-09-20 07:48:04+01 |  28.19737 | 113.006795 |  71
      11 |  20070920074804   | 2007-09-20 08:07:09+01 | 28.197685 | 113.006792 |  87
      11 |  20070920074804   | 2007-09-20 08:07:10+01 | 28.197685 |  113.00679 |  87
      11 |  2007092007480402 | 2007-09-20 14:03:50+01 | 28.197342 | 113.007422 |  62
      11 |  2007092007480402 | 2007-09-20 14:04:59+01 | 28.197108 |  113.00734 |  62
      11 |  2007092007480402 | 2007-09-20 14:05:01+01 | 28.197088 |  113.00727 |  62
请注意我是如何为新行程分配session_id的,因为两者之间的时间超过30分钟

我如何在postgres中修改或更改我的原始GPS表轨迹

编辑

答:@GMB答案中的第一个查询是有效的,但是,它在new_session_id列中为每一行提供了一个新的session_id

预期结果:

我们的想法是通过旧的会话id+01为新兴的旅行提供一个新的id。如果遇到另一个新的行程,则应为其分配旧的会话id+02,以此类推

B:第二个带更新选项的查询包含语法错误:

update trajectories t
from (
    select 
        t.*,
        case when sum(is_gap) over(partition by session_id order by timestamp) > 0
            then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
            else session_id
        end new_session_id
    from (
        select
            t.*,
            (timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
        from trajectories t
    ) t
) t1
set session_id = t1.new_session_id
where t1.session_id = t.session_id and t1.timestamp = t.timestamp

ERROR:  syntax error at or near "from"
LINE 2: from (
您可以使用lag(一个累计总和)来标识分段,然后使用某种方式来调整会话id:


这是一个空白和孤岛问题。您希望检测时间戳差异大于30分钟的连续行,然后相应地更改会话id

一个选项是使用lag,然后是间隙的累积计数-然后可以使用该信息计算新的会话id:

如果需要,可以将其转换为update语句:

update trajectories t
set session_id = t1.new_session_id
from (
    select 
        t.*,
        case when sum(is_gap) over(partition by session_id order by timestamp) > 0
            then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
            else session_id
        end new_session_id
    from (
        select
            t.*,
            (timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
        from trajectories t
    ) t
) t1
where t1.session_id = t.session_id and t1.timestamp = t.timestamp

谢谢你的回答。我真的很想用update选项来回答这个问题,但是它需要像我的问题编辑中所说的那样重新阅读。@someone:my bad,Postgres希望set子句在from子句之前。已修复。获取SQL语法错误:错误:关系t不存在第9行:从t@someone . . . 这是您的表名。在这个db提琴中,我说明了错误。看来这个答案考虑了过滤要求,30分钟间隔我不明白为什么postgres总是抱怨子查询别名。你能看看我上面评论中的db提琴吗?@someone。哎呀,那应该是一个窗口函数。我修复了查询并添加了一个dbfiddle。
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
| user_id |   session_id   |       timestamp        |    lat    |    lon     | alt | is_gap |  new_session_id  |
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
|      11 | 20070920074804 | 2007-09-20 07:48:04+01 | 28.19737  | 113.006795 |  71 |        |   20070920074804 |
|      11 | 20070920074804 | 2007-09-20 08:07:09+01 | 28.197685 | 113.006792 |  87 |        |   20070920074804 |
|      11 | 20070920074804 | 2007-09-20 08:07:10+01 | 28.197685 | 113.00679  |  87 |      1 | 2007092007480401 |
|      11 | 20070920074804 | 2007-09-20 14:03:50+01 | 28.197342 | 113.007422 |  62 |      1 | 2007092007480401 |
|      11 | 20070920074804 | 2007-09-20 14:04:59+01 | 28.197108 | 113.00734  |  62 |      1 | 2007092007480401 |
|      11 | 20070920074804 | 2007-09-20 14:05:01+01 | 28.197088 | 113.00727  |  62 |      1 | 2007092007480401 |
+---------+----------------+------------------------+-----------+------------+-----+--------+------------------+
update trajectories t
from (
    select 
        t.*,
        case when sum(is_gap) over(partition by session_id order by timestamp) > 0
            then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
            else session_id
        end new_session_id
    from (
        select
            t.*,
            (timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
        from trajectories t
    ) t
) t1
set session_id = t1.new_session_id
where t1.session_id = t.session_id and t1.timestamp = t.timestamp

ERROR:  syntax error at or near "from"
LINE 2: from (
select (case when grp >= 1 then session_id * 100 + grp
             else session_id
        end) as new_session_id,
       t.*
from (select t.*,
             count(*) filter (where prev_ts < timestamp - interval '30 minute') over (partition by session_id, order by timestamp) as grp
      from (select t.*, 
                   lag(timestamp) over (partition by session_id order by timestamp) as prev_ts
            from trajectories t
           ) t
     ) t;
select 
    t.*,
    case when sum(is_gap) over(partition by session_id order by timestamp) > 0
        then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
        else session_id
    end new_session_id
from (
    select
        t.*,
        (timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
    from trajectories t
) t
update trajectories t
set session_id = t1.new_session_id
from (
    select 
        t.*,
        case when sum(is_gap) over(partition by session_id order by timestamp) > 0
            then session_id * 100 + sum(is_gap) over(partition by session_id order by timestamp)
            else session_id
        end new_session_id
    from (
        select
            t.*,
            (timestamp > lag(timestamp) over(partition by session_id order by timestamp))::int is_gap
        from trajectories t
    ) t
) t1
where t1.session_id = t.session_id and t1.timestamp = t.timestamp