Pandas 计算列中元素在时间戳上出现的次数
对于PandasPandas 计算列中元素在时间戳上出现的次数,pandas,dataframe,time,count,Pandas,Dataframe,Time,Count,对于Pandas数据帧中的给定行,我需要计算当前列值的次数,例如“destination\u address\u IP”在过去(比如)2秒内使用“Time\u stamp”列出现的次数,并将这些值放入新的列“count”。通过重复移动数据帧,您可以按如下方式进行操作。基本假设是,数据帧按时间戳列排序: # define the threshold in milliseconds (2 seconds) time_threshold= 2000000 df['ip_count']=1 df_shi
数据帧中的给定行,我需要计算当前列值的次数,例如“destination\u address\u IP”在过去(比如)2秒内使用“Time\u stamp”列出现的次数,并将这些值放入新的列“count”。通过重复移动数据帧,您可以按如下方式进行操作。基本假设是,数据帧按时间戳列排序:
# define the threshold in milliseconds (2 seconds)
time_threshold= 2000000
df['ip_count']=1
df_shifted= df
# loop over the dataframe and shift it by one row
# until the time_threshold is violated for all rows
while True:
# shift the copy of the dataframe
df_shifted= df_shifted.shift(1)
# check if time range is ok
ser_time_diff= (df['Time_stamp'] - df_shifted['Time_stamp'])
ser_in_time= ser_time_diff.dt.microseconds + ser_time_diff.dt.seconds * 1000000 < time_threshold
if ser_in_time.any():
# there are still rows left, where the shifted
# frame's timestamp lies within the threshold
# so we need to count the matches for those rows
# if there are any
ser_match= ser_in_time & (df['destination_address_IP'] == df_shifted['destination_address_IP'])
df['ip_count']+= ser_match.astype('int')
else:
# none of the rows of the shifted df was within
# the threshold of the original df
# so further shifts will not change the result
# anymore
break
df
它是根据以下数据生成的:
import io
import pandas as pd
raw=\
"""Time_stamp destination_address_IP
2019-09-17T19:20:45.093209 157.111.73.31
2019-09-17T19:20:45.297932 127.0.0.1
2019-09-17T19:20:45.750725 157.111.73.31
2019-09-17T19:20:46.787009 192.168.21.15
2019-09-17T19:20:47.601051 52.18.181.18
2019-09-17T19:20:47.863428 52.18.181.17
2019-09-17T19:20:48.418591 52.18.181.18
2019-09-17T19:20:48.596764 52.18.181.17
2019-09-17T19:20:49.057553 192.168.21.15
2019-09-17T19:20:49.153256 192.168.21.15
2019-09-17T19:20:49.712312 127.0.0.1
2019-09-17T19:20:50.000119 52.18.181.17
2019-09-17T19:20:50.248562 52.18.181.18
2019-09-17T19:20:50.603783 52.18.181.18
2019-09-17T19:20:50.921631 52.18.181.17
2019-09-17T19:20:51.366193 52.18.181.18
2019-09-17T19:20:51.528611 52.18.181.18
2019-09-17T19:20:51.773429 131.53.97.59
2019-09-17T19:20:52.618215 192.168.21.15
2019-09-17T19:20:52.936181 52.18.181.18
"""
df= pd.read_csv(
io.StringIO(raw),
sep='\s{2,}', dtype={
'Time_stamp': 'datetime64',
'destination_address_IP': 'str'},
engine='python')
你能提供样品数据和预期结果吗?欢迎访问。请检查并创建一个。所以这不是一个免费的代码编写服务,也不是一个教程网站。
import io
import pandas as pd
raw=\
"""Time_stamp destination_address_IP
2019-09-17T19:20:45.093209 157.111.73.31
2019-09-17T19:20:45.297932 127.0.0.1
2019-09-17T19:20:45.750725 157.111.73.31
2019-09-17T19:20:46.787009 192.168.21.15
2019-09-17T19:20:47.601051 52.18.181.18
2019-09-17T19:20:47.863428 52.18.181.17
2019-09-17T19:20:48.418591 52.18.181.18
2019-09-17T19:20:48.596764 52.18.181.17
2019-09-17T19:20:49.057553 192.168.21.15
2019-09-17T19:20:49.153256 192.168.21.15
2019-09-17T19:20:49.712312 127.0.0.1
2019-09-17T19:20:50.000119 52.18.181.17
2019-09-17T19:20:50.248562 52.18.181.18
2019-09-17T19:20:50.603783 52.18.181.18
2019-09-17T19:20:50.921631 52.18.181.17
2019-09-17T19:20:51.366193 52.18.181.18
2019-09-17T19:20:51.528611 52.18.181.18
2019-09-17T19:20:51.773429 131.53.97.59
2019-09-17T19:20:52.618215 192.168.21.15
2019-09-17T19:20:52.936181 52.18.181.18
"""
df= pd.read_csv(
io.StringIO(raw),
sep='\s{2,}', dtype={
'Time_stamp': 'datetime64',
'destination_address_IP': 'str'},
engine='python')