在python中的子数据集中搜索
我想在在python中的子数据集中搜索,python,pandas,numpy,select,group-by,Python,Pandas,Numpy,Select,Group By,我想在df中的每个子数据集中搜索： id timestamp data gradient Start timestamp 2020-01-15 06:12:49.213 40250 2020-01-15 06:12:49.213 20.0 0.00373 NaN 2020-01-15 0
df id timestamp data gradient Start
timestamp
2020-01-15 06:12:49.213 40250 2020-01-15 06:12:49.213 20.0 0.00373 NaN
2020-01-15 06:12:49.313 40251 2020-01-15 06:12:49.313 19.5 0.00354 0.0
2020-01-15 08:05:10.083 40256 2020-01-15 08:05:10.083 20.0 0.00020 1.0
2020-01-15 08:05:10.183 40257 2020-01-15 08:05:10.183 20.5 -0.00440 0.0
...
2020-01-31 09:01:50.993 40310 2020-01-31 09:01:50.993 21.0 0.55473 1.0
2020-01-31 09:01:51.093 40311 2020-01-31 09:01:51.093 21.5 0.00589 0.0
...
df id timestamp data gradient Start
timestamp
2020-01-15 06:12:49.213 40250 2020-01-15 06:12:49.213 20.0 0.00373 NaN
2020-01-15 06:12:49.313 40251 2020-01-15 06:12:49.313 19.5 0.00354 0.0
2020-01-15 08:05:10.083 40256 2020-01-15 08:05:10.083 20.0 0.00020 1.0
2020-01-15 08:05:10.183 40257 2020-01-15 08:05:10.183 20.5 -0.00440 0.0
...
2020-01-31 09:01:50.993 40310 2020-01-31 09:01:50.993 21.0 0.55473 1.0
2020-01-31 09:01:51.093 40311 2020-01-31 09:01:51.093 21.5 0.00589 0.0
...
Start==1Start==1gradient>0.0003end\u timestart==1start\u time数据的平均值，以获得如下表：
start_time end_time Average
2020-01-15 08:05:10.083 2020-01-15 08:05:23.273 35(for example)
...
编辑：
可复制数据帧：
d = {'timestamp':["2020-01-15 06:12:49.213", "2020-01-15 06:12:49.313", "2020-01-15 08:05:10.083", "2020-01-15 08:05:10.183", "2020-01-15 09:01:50.993", "2020-01-15 09:01:51.093", "2020-01-15 09:51:01.890", "2020-01-15 09:51:01.990", "2020-01-15 10:40:59.657", "2020-01-15 10:40:59.757", "2020-01-15 10:42:55.693", "2020-01-15 10:42:55.793", "2020-01-15 10:45:35.767", "2020-01-15 10:45:35.867", "2020-01-15 10:45:46.770", "2020-01-15 10:45:46.870", "2020-01-15 10:47:19.783", "2020-01-15 10:47:19.883", "2020-01-15 10:47:22.787"],
'data': [20.0, 19.5, 20.0, 20.5, 21.0, 21.5, 22.0, 22.5, 23.0, 23.5, 23.0, 22.5, 23.0, 23.5, 24.0, 24.5, 25.0, 25.5, 26],
'gradient': [NaN, NaN, 0.000000, 0.000148, 0.000294, 0.000294, 0.000339, 0.000339, 0.000334, 0.000334, 0.000000, -0.008618, 0.000000, 0.006247, 0.090884, 0.090884, 0.010751, 0.010751, 0.332889],
'Start': [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,]
}
df = pd.DataFrame(d)
start_time end_time Average
2020-01-15 08:05:10.083 2020-01-15 09:01:51.093 20.75 = average of (20.0, 20.5, 21.0, 21.5)
2020-01-15 10:45:35.767 2020-01-15 10:45:35.767 23.00 = average of (23.0)
可再现数据帧的预期输出：
d = {'timestamp':["2020-01-15 06:12:49.213", "2020-01-15 06:12:49.313", "2020-01-15 08:05:10.083", "2020-01-15 08:05:10.183", "2020-01-15 09:01:50.993", "2020-01-15 09:01:51.093", "2020-01-15 09:51:01.890", "2020-01-15 09:51:01.990", "2020-01-15 10:40:59.657", "2020-01-15 10:40:59.757", "2020-01-15 10:42:55.693", "2020-01-15 10:42:55.793", "2020-01-15 10:45:35.767", "2020-01-15 10:45:35.867", "2020-01-15 10:45:46.770", "2020-01-15 10:45:46.870", "2020-01-15 10:47:19.783", "2020-01-15 10:47:19.883", "2020-01-15 10:47:22.787"],
'data': [20.0, 19.5, 20.0, 20.5, 21.0, 21.5, 22.0, 22.5, 23.0, 23.5, 23.0, 22.5, 23.0, 23.5, 24.0, 24.5, 25.0, 25.5, 26],
'gradient': [NaN, NaN, 0.000000, 0.000148, 0.000294, 0.000294, 0.000339, 0.000339, 0.000334, 0.000334, 0.000000, -0.008618, 0.000000, 0.006247, 0.090884, 0.090884, 0.010751, 0.010751, 0.332889],
'Start': [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,]
}
df = pd.DataFrame(d)
start_time end_time Average
2020-01-15 08:05:10.083 2020-01-15 09:01:51.093 20.75 = average of (20.0, 20.5, 21.0, 21.5)
2020-01-15 10:45:35.767 2020-01-15 10:45:35.767 23.00 = average of (23.0)
我相信你需要：
df['g'] = df['Start'].cumsum()
df['m'] = df['gradient'].gt(0.0003)
#filter first group - rows before first 1
df1 = df[df['g'].ne(0)].copy()
#filter rows to first True in column m
df1 = df1[df1.groupby('g')['m'].cumsum().eq(0)]
#named aggregation
df2 = df1.groupby('g').agg(start_time=('timestamp','first'),
end_time=('timestamp','last'),
Average=('data','mean')).reset_index(drop=True)
print (df2)
start_time end_time Average
0 2020-01-15 08:05:10.083 2020-01-15 09:01:51.093 20.75
1 2020-01-15 10:45:35.767 2020-01-15 10:45:35.767 23.00
我相信你需要：
df['g'] = df['Start'].cumsum()
df['m'] = df['gradient'].gt(0.0003)
#filter first group - rows before first 1
df1 = df[df['g'].ne(0)].copy()
#filter rows to first True in column m
df1 = df1[df1.groupby('g')['m'].cumsum().eq(0)]
#named aggregation
df2 = df1.groupby('g').agg(start_time=('timestamp','first'),
end_time=('timestamp','last'),
Average=('data','mean')).reset_index(drop=True)
print (df2)
start_time end_time Average
0 2020-01-15 08:05:10.083 2020-01-15 09:01:51.093 20.75
1 2020-01-15 10:45:35.767 2020-01-15 10:45:35.767 23.00
是否可能更改数据？我认为至少有一组符合条件。@jezrael请参见编辑后的问题。那是橡木的吗？还请注意，我将阈值更改为0.0003
，以适应可再现的数据帧。谢谢。我想是mcve，意思是5-8行。哪些行用于编辑数据帧的平均值？预期输出是什么？平均值是否包含在内？好的，请给我一些时间。没问题；）是否可能更改数据？我认为至少有一组符合条件。@jezrael请参见编辑后的问题。那是橡木的吗？还请注意，我将阈值更改为0.0003
，以适应可再现的数据帧。谢谢。我想是mcve，意思是5-8行。哪些行用于编辑数据帧的平均值？预期输出是什么？平均值是否包含在内？好的，请给我一些时间。没问题；）嗨，我刚刚添加了可再现数据帧的预期输出。我认为平均值是独占的（>）。很抱歉，应该大于-0.0003，小于0
。好的，我知道了。对于第一个df['m']=df['gradient'].gt（0.0003）
更改为df['m']=df['gradient']。介于（-0.0003，0，包括=False）
我注意到结束时间和下一个开始时间都是背靠背的？应该是这样吗？是的，我认为这很有道理。请允许我再核对一下数据。非常感谢您的帮助：）嗨，我刚刚添加了可再现数据帧的预期输出。我认为平均值是独占的（>）。很抱歉，应该大于-0.0003，小于0
。好的，我知道了。对于第一个df['m']=df['gradient'].gt（0.0003）
更改为df['m']=df['gradient']。介于（-0.0003，0，包括=False）
我注意到结束时间和下一个开始时间都是背靠背的？应该是这样吗？是的，我认为这很有道理。请允许我再核对一下数据。非常感谢您的帮助：）