Python中datetime列中多个事件的持续时间

我有来自多个运动传感器的以下示例数据（multiple_sensors.csv）：

sensorid,date_time,value
303,2012-06-25 11:15:35,0
404,2012-06-25 11:15:35,0
101,2012-06-25 11:15:35,0
202,2012-06-25 11:15:35,0
303,2012-06-25 11:15:36,0
404,2012-06-25 11:15:36,0
101,2012-06-25 11:15:36,0
202,2012-06-25 11:15:36,1
303,2012-06-25 11:15:37,0
404,2012-06-25 11:15:37,0
101,2012-06-25 11:15:37,0
202,2012-06-25 11:15:37,1
303,2012-06-25 11:15:38,0
404,2012-06-25 11:15:38,0
101,2012-06-25 11:15:38,0
202,2012-06-25 11:15:38,0
303,2012-06-25 11:15:39,0
404,2012-06-25 11:15:39,1
101,2012-06-25 11:15:39,0
202,2012-06-25 11:15:39,0
303,2012-06-25 11:15:40,0
404,2012-06-25 11:15:40,1
101,2012-06-25 11:15:40,0
202,2012-06-25 11:15:40,0
303,2012-06-25 11:15:41,1
404,2012-06-25 11:15:41,0
101,2012-06-25 11:15:41,0
202,2012-06-25 11:15:41,0
303,2012-06-25 11:15:42,1
404,2012-06-25 11:15:42,0
101,2012-06-25 11:15:42,0
202,2012-06-25 11:15:42,0
303,2012-06-25 11:15:43,1
404,2012-06-25 11:15:43,0
101,2012-06-25 11:15:43,0
202,2012-06-25 11:15:43,0
303,2012-06-25 11:15:44,0

我需要按发生顺序返回每个运动传感器事件的id和持续时间（请参阅）。值列确定是否触发运动（1-表示运动已触发，0-表示无运动），日期时间列指示运动开始或结束的时间

目前，我使用下面的单个运动传感器（single_sensor.csv）提取id和持续时间（请参阅）

对于涉及单个传感器的代码，我遵循了下面的示例（）

对于单个传感器，如何使用多个传感器.csv来扩展此功能以实现预期输出：

import pandas as pd
df = pd.read_csv('single_censor.csv')
df['date_time'] = pd.to_datetime(df['date_time'])

# Assume that your data format first value=0 ignore, start value=1 end value=0
selected_rows = df['value'] != df['value'].shift(1)
selected_rows[0] = False

df2 = df[selected_rows].copy()

df2['start'] = df2['date_time']
df2['end'] = df2['date_time'].shift(-1)
df2.drop(['date_time'], axis=1, inplace=True)

df3 = df2[df2['value'] == 1].copy()

df3['duration'] = df3['end'] - df3['start']
df3.drop('value', axis=1, inplace=True)

输出

    sensorid    start   end duration
1   202 2012-06-25 00:01:08 2012-06-25 00:01:10 00:00:02
5   202 2012-06-25 00:02:13 2012-06-25 00:02:16 00:00:03
10  202 2012-06-25 00:03:41 2012-06-25 00:03:44 00:00:03
15  202 2012-06-25 00:05:12 2012-06-25 00:05:14 00:00:02
19  202 2012-06-25 00:06:20 2012-06-25 00:06:22 00:00:02

    sensorid               start                 end duration
7        202 2012-06-25 11:15:36 2012-06-25 11:15:38 00:00:02
17       404 2012-06-25 11:15:39 2012-06-25 11:15:41 00:00:02
24       303 2012-06-25 11:15:41 2012-06-25 11:15:44 00:00:03

   sensor_id               start                 end  duration
0        202 2012-06-25 00:11:47 2012-06-25 00:11:49         2

   sensor_id  duration2
1        202        4.0
3        404        5.0
5        202        6.0
7        101        4.0

   index  sensor_id               start                 end duration
0      1        202 2020-06-25 00:11:43 2020-06-25 00:11:49 00:00:06
1      3        404 2020-06-25 00:11:51 2020-06-25 00:11:57 00:00:06
2      5        202 2020-06-25 00:11:58 2020-06-25 00:12:21 00:00:23
3      7        101 2020-06-25 00:12:21 2020-06-25 00:12:34 00:00:13

多个传感器：

import pandas as pd
df = pd.read_csv('multiple_sensors.csv')
df['date_time'] = pd.to_datetime(df['date_time'])
df2 = df.sort_values(['sensorid', 'date_time'])

selected_rows = df2['value'] != df2['value'].shift(1)
selected_rows[0] = False

df3 = df2[selected_rows].copy()
df3['start'] = df3['date_time']
df3['end'] = df3['date_time'].shift(-1)
df3.drop(['date_time'], axis=1, inplace=True)

df4 = df3[df3['value'] == 1].copy()
df4['duration'] = df4['end'] - df4['start']
df4.drop('value', axis=1, inplace=True)
df4.sort_values('start') 

输出

    sensorid    start   end duration
1   202 2012-06-25 00:01:08 2012-06-25 00:01:10 00:00:02
5   202 2012-06-25 00:02:13 2012-06-25 00:02:16 00:00:03
10  202 2012-06-25 00:03:41 2012-06-25 00:03:44 00:00:03
15  202 2012-06-25 00:05:12 2012-06-25 00:05:14 00:00:02
19  202 2012-06-25 00:06:20 2012-06-25 00:06:22 00:00:02

    sensorid               start                 end duration
7        202 2012-06-25 11:15:36 2012-06-25 11:15:38 00:00:02
17       404 2012-06-25 11:15:39 2012-06-25 11:15:41 00:00:02
24       303 2012-06-25 11:15:41 2012-06-25 11:15:44 00:00:03

   sensor_id               start                 end  duration
0        202 2012-06-25 00:11:47 2012-06-25 00:11:49         2

   sensor_id  duration2
1        202        4.0
3        404        5.0
5        202        6.0
7        101        4.0

   index  sensor_id               start                 end duration
0      1        202 2020-06-25 00:11:43 2020-06-25 00:11:49 00:00:06
1      3        404 2020-06-25 00:11:51 2020-06-25 00:11:57 00:00:06
2      5        202 2020-06-25 00:11:58 2020-06-25 00:12:21 00:00:23
3      7        101 2020-06-25 00:12:21 2020-06-25 00:12:34 00:00:13

删除重叠时间：

data = [
    (202, pd.to_datetime('2012-06-25 00:11:47'),
     pd.to_datetime('2012-06-25 00:11:49'), 2),
    (404, pd.to_datetime('2012-06-25 00:11:48'),
     pd.to_datetime('2012-06-25 00:11:50'), 2)
]
df = pd.DataFrame(data, columns=['sensor_id', 'start', 'end', 'duration'])

df['end_shift'] = df['end'].shift().fillna(pd.to_datetime('1971-01-01'))
df.loc[0, 'end_shift'] = pd.to_datetime('1971-01-01')
df[df['start'] >= df['end_shift']].drop('end_shift', axis=1)

输出

    sensorid    start   end duration
1   202 2012-06-25 00:01:08 2012-06-25 00:01:10 00:00:02
5   202 2012-06-25 00:02:13 2012-06-25 00:02:16 00:00:03
10  202 2012-06-25 00:03:41 2012-06-25 00:03:44 00:00:03
15  202 2012-06-25 00:05:12 2012-06-25 00:05:14 00:00:02
19  202 2012-06-25 00:06:20 2012-06-25 00:06:22 00:00:02

    sensorid               start                 end duration
7        202 2012-06-25 11:15:36 2012-06-25 11:15:38 00:00:02
17       404 2012-06-25 11:15:39 2012-06-25 11:15:41 00:00:02
24       303 2012-06-25 11:15:41 2012-06-25 11:15:44 00:00:03

   sensor_id               start                 end  duration
0        202 2012-06-25 00:11:47 2012-06-25 00:11:49         2

   sensor_id  duration2
1        202        4.0
3        404        5.0
5        202        6.0
7        101        4.0

   index  sensor_id               start                 end duration
0      1        202 2020-06-25 00:11:43 2020-06-25 00:11:49 00:00:06
1      3        404 2020-06-25 00:11:51 2020-06-25 00:11:57 00:00:06
2      5        202 2020-06-25 00:11:58 2020-06-25 00:12:21 00:00:23
3      7        101 2020-06-25 00:12:21 2020-06-25 00:12:34 00:00:13

组持续时间：

data = [
(202, pd.to_datetime('2020-06-25 00:11:43'), pd.to_datetime('2020-06-25 00:11:45'),2), 
(202, pd.to_datetime('2020-06-25 00:11:47'), pd.to_datetime('2020-06-25 00:11:49'),2),
(404, pd.to_datetime('2020-06-25 00:11:51'), pd.to_datetime('2020-06-25 00:11:54'),3),
(404, pd.to_datetime('2020-06-25 00:11:55'), pd.to_datetime('2020-06-25 00:11:57'),2),
(202, pd.to_datetime('2020-06-25 00:11:58'), pd.to_datetime('2020-06-25 00:12:01'),3),
(202, pd.to_datetime('2020-06-25 00:12:18'), pd.to_datetime('2020-06-25 00:12:21'),3),
(101, pd.to_datetime('2020-06-25 00:12:21'), pd.to_datetime('2020-06-25 00:12:23'),2),
(101, pd.to_datetime('2020-06-25 00:12:32'), pd.to_datetime('2020-06-25 00:12:34'),2),
]
df=pd.DataFrame(data, columns=['sensor_id', 'start', 'end', 'duration'])

df['id'] = df['sensor_id'].shift(-1)
df['cumsum'] = df['duration'].cumsum()
df2 = df[df['id'] != df['sensor_id']].copy()
df2['duration2'] = df2['cumsum'] - df2['cumsum'].shift().fillna(0)
df2[['sensor_id', 'duration2']]

输出

    sensorid    start   end duration
1   202 2012-06-25 00:01:08 2012-06-25 00:01:10 00:00:02
5   202 2012-06-25 00:02:13 2012-06-25 00:02:16 00:00:03
10  202 2012-06-25 00:03:41 2012-06-25 00:03:44 00:00:03
15  202 2012-06-25 00:05:12 2012-06-25 00:05:14 00:00:02
19  202 2012-06-25 00:06:20 2012-06-25 00:06:22 00:00:02

    sensorid               start                 end duration
7        202 2012-06-25 11:15:36 2012-06-25 11:15:38 00:00:02
17       404 2012-06-25 11:15:39 2012-06-25 11:15:41 00:00:02
24       303 2012-06-25 11:15:41 2012-06-25 11:15:44 00:00:03

   sensor_id               start                 end  duration
0        202 2012-06-25 00:11:47 2012-06-25 00:11:49         2

   sensor_id  duration2
1        202        4.0
3        404        5.0
5        202        6.0
7        101        4.0

   index  sensor_id               start                 end duration
0      1        202 2020-06-25 00:11:43 2020-06-25 00:11:49 00:00:06
1      3        404 2020-06-25 00:11:51 2020-06-25 00:11:57 00:00:06
2      5        202 2020-06-25 00:11:58 2020-06-25 00:12:21 00:00:23
3      7        101 2020-06-25 00:12:21 2020-06-25 00:12:34 00:00:13

要求从一开始就不明确。所有原始计算的持续时间都将被丢弃，并重新计算新的持续时间。如果要求明确就更好了。解决方案将被短路

data = [
(202, pd.to_datetime('2020-06-25 00:11:43'), pd.to_datetime('2020-06-25 00:11:45'),2), 
(202, pd.to_datetime('2020-06-25 00:11:47'), pd.to_datetime('2020-06-25 00:11:49'),2),
(404, pd.to_datetime('2020-06-25 00:11:51'), pd.to_datetime('2020-06-25 00:11:54'),3),
(404, pd.to_datetime('2020-06-25 00:11:55'), pd.to_datetime('2020-06-25 00:11:57'),2),
(202, pd.to_datetime('2020-06-25 00:11:58'), pd.to_datetime('2020-06-25 00:12:01'),3),
(202, pd.to_datetime('2020-06-25 00:12:18'), pd.to_datetime('2020-06-25 00:12:21'),3),
(101, pd.to_datetime('2020-06-25 00:12:21'), pd.to_datetime('2020-06-25 00:12:23'),2),
(101, pd.to_datetime('2020-06-25 00:12:32'), pd.to_datetime('2020-06-25 00:12:34'),2),
]
df=pd.DataFrame(data, columns=['sensor_id', 'start', 'end', 'duration'])

df['id1'] = df['sensor_id'].shift(-1)
df['id2'] = df['sensor_id'].shift(1)

df2 = df[df['id1'] != df['sensor_id']].copy().reset_index()
df2['start'] = df[df['id2'] != df['sensor_id']].reset_index()['start']

df2['duration'] = df2['end'] - df2['start']
df2.drop(['id1', 'id2'], axis=1, inplace=True) 
df2

输出

    sensorid    start   end duration
1   202 2012-06-25 00:01:08 2012-06-25 00:01:10 00:00:02
5   202 2012-06-25 00:02:13 2012-06-25 00:02:16 00:00:03
10  202 2012-06-25 00:03:41 2012-06-25 00:03:44 00:00:03
15  202 2012-06-25 00:05:12 2012-06-25 00:05:14 00:00:02
19  202 2012-06-25 00:06:20 2012-06-25 00:06:22 00:00:02

    sensorid               start                 end duration
7        202 2012-06-25 11:15:36 2012-06-25 11:15:38 00:00:02
17       404 2012-06-25 11:15:39 2012-06-25 11:15:41 00:00:02
24       303 2012-06-25 11:15:41 2012-06-25 11:15:44 00:00:03

   sensor_id               start                 end  duration
0        202 2012-06-25 00:11:47 2012-06-25 00:11:49         2

   sensor_id  duration2
1        202        4.0
3        404        5.0
5        202        6.0
7        101        4.0

   index  sensor_id               start                 end duration
0      1        202 2020-06-25 00:11:43 2020-06-25 00:11:49 00:00:06
1      3        404 2020-06-25 00:11:51 2020-06-25 00:11:57 00:00:06
2      5        202 2020-06-25 00:11:58 2020-06-25 00:12:21 00:00:23
3      7        101 2020-06-25 00:12:21 2020-06-25 00:12:34 00:00:13

IIUC

让我们试试这个：

def f(df):
    a = (df['value'] != 1).cumsum().mask(df['value'] == 1)
    df['value group'] = a.bfill()

    df_final = df.groupby('value group').filter(lambda x: set(x['value']) == set([1,0]))\
           .groupby('value group')['date_time'].agg(['first','last'])\
           .rename(columns={'first':'start','last':'end'})\
           .reset_index()
    if df_final.shape[0] == 0:
        return
    df_final['id'] = df['sensorid']
    df_final['duration'] = df_final['end'].values - df_final['start']
    df_final['duration'] = df_final['duration'].dt.total_seconds().astype(int)
    return df_final

df_out = df.groupby('sensorid').apply(f).reset_index().drop(['level_1', 'value group', 'id'], axis=1)
df_out = df_out.sort_values('start')
df_out

输出：

   sensorid               start                 end  duration
0       202 2012-06-25 11:15:36 2012-06-25 11:15:38         2
1       303 2012-06-25 11:15:41 2012-06-25 11:15:44         3
2       404 2012-06-25 11:15:39 2012-06-25 11:15:41         2

---

注意：这可能需要一个更健壮的测试用例。但是，在groupby'sensorid'调用的自定义函数中使用前面的逻辑。

列值是多少？你认为他们什么时候是开始时间和停止时间？非常感谢史葛。该解决方案通过按sensorid分组实现了我所需的功能，这意味着它首先找到202的所有持续时间，然后再转到303，最后是404。正如我在问题中提到的，我如何按照发生顺序实现相同的功能，即基于“日期时间”而不是传感器ID。如有必要，我可以与您共享更长的数据集。再次感谢，这正是我想要的。谢谢Scott，很好的解决方案Pramote，很好用。正如我在文章中提到的，如何根据事件发生的顺序（即“日期时间”而不是sensorid）获得每个传感器事件的持续时间。目前，解决方案首先迭代每个传感器ID，然后再进入下一个传感器ID。如有必要，我很乐意提供更多信息。再次感谢.df4.排序值（“开始”）作为我问题的后续，我的输出有一些重叠，如下所示：

sensor\u id，start，end，duration

2022012-06-25 00:11:472012-06-25 00:11:49,24042012-06-25 00:11:482012-06-25 00:11:50,2结束时间[2012-06-25 00:11:49]第一行的开始时间应小于第二行的开始时间[2012-06-25 00:11:48]如何删除重叠？要调整时间还是删除重叠？如果要调整时间，请选择开始时间还是结束时间？您的数据帧必须是带持续时间的输出，而不是原始数据帧。只需删除列end_shift，您将获得删除重叠行的输出数据帧。