Python 对数据帧中的连续数据进行计数,并在出现这种情况时获取索引
我有一个带有整数列名的Python 对数据帧中的连续数据进行计数,并在出现这种情况时获取索引,python,pandas,itertools,Python,Pandas,Itertools,我有一个带有整数列名的pandas.DataFrame,它有0和1。输入的一个示例: 12 13 14 15 1 0 0 1 0 2 0 0 1 1 3 1 0 0 1 4 1 1 0 1 5 1 1 1 0 6 0 0 1 0 7 0 0 1 1 8 1 1 0 1 9 0 0 1 1 10 0 0 1 1 11 1
pandas.DataFrame
,它有0和1。输入的一个示例:
12 13 14 15
1 0 0 1 0
2 0 0 1 1
3 1 0 0 1
4 1 1 0 1
5 1 1 1 0
6 0 0 1 0
7 0 0 1 1
8 1 1 0 1
9 0 0 1 1
10 0 0 1 1
11 1 1 0 1
12 1 1 1 1
13 1 1 1 1
14 1 0 1 1
15 0 0 1 1
我需要计算长度/和大于等于2的所有连续的索引,遍历列,并返回出现连续索引数组的索引(开始、结束)
首选的输出是3D数据帧,其中子列“count”和“index”表示输入中的整数列名
示例输出如下所示:
12 13 14 15
count indices count indices count indices count indices
3 (3,5) 2 (4,5) 2 (1,2) 3 (2,4)
4 (11,14) 3 (11,13) 3 (5,7) 9 (7,15)
2 (9,10)
4 (12,15)
我想应该用
itertools.groupby
解决这个问题,但是仍然无法找出如何将它应用到这样的问题,在这个问题中,groupby
结果及其索引都被提取出来。这里有一种计算所需运行长度的方法:
代码:
def min_run_length(series):
terminal = pd.Series([0])
diffs = pd.concat([terminal, series, terminal]).diff()
starts = np.where(diffs == 1)
ends = np.where(diffs == -1)
return [(e-s, (s, e-1)) for s, e in zip(starts[0], ends[0])
if e - s >= 2]
df = pd.read_fwf(StringIO(u"""
12 13 14 15
0 0 1 0
0 0 1 1
1 0 0 1
1 1 0 1
1 1 1 0
0 0 1 0
0 0 1 1
1 1 0 1
0 0 1 1
0 0 1 1
1 1 0 1
1 1 1 1
1 1 1 1
1 0 1 1
0 0 1 1"""), header=1)
print(df.dtypes)
indices = {cname: min_run_length(df[cname]) for cname in df.columns}
print(indices)
{
u'12': [(3, (3, 5)), (4, (11, 14))],
u'13': [(2, (4, 5)), (3, (11, 13))],
u'14': [(2, (1, 2)), (3, (5, 7)), (2, (9, 10)), (4, (12, 15))]
u'15': [(3, (2, 4)), (9, (7, 15))],
}
测试代码:
def min_run_length(series):
terminal = pd.Series([0])
diffs = pd.concat([terminal, series, terminal]).diff()
starts = np.where(diffs == 1)
ends = np.where(diffs == -1)
return [(e-s, (s, e-1)) for s, e in zip(starts[0], ends[0])
if e - s >= 2]
df = pd.read_fwf(StringIO(u"""
12 13 14 15
0 0 1 0
0 0 1 1
1 0 0 1
1 1 0 1
1 1 1 0
0 0 1 0
0 0 1 1
1 1 0 1
0 0 1 1
0 0 1 1
1 1 0 1
1 1 1 1
1 1 1 1
1 0 1 1
0 0 1 1"""), header=1)
print(df.dtypes)
indices = {cname: min_run_length(df[cname]) for cname in df.columns}
print(indices)
{
u'12': [(3, (3, 5)), (4, (11, 14))],
u'13': [(2, (4, 5)), (3, (11, 13))],
u'14': [(2, (1, 2)), (3, (5, 7)), (2, (9, 10)), (4, (12, 15))]
u'15': [(3, (2, 4)), (9, (7, 15))],
}
结果:
def min_run_length(series):
terminal = pd.Series([0])
diffs = pd.concat([terminal, series, terminal]).diff()
starts = np.where(diffs == 1)
ends = np.where(diffs == -1)
return [(e-s, (s, e-1)) for s, e in zip(starts[0], ends[0])
if e - s >= 2]
df = pd.read_fwf(StringIO(u"""
12 13 14 15
0 0 1 0
0 0 1 1
1 0 0 1
1 1 0 1
1 1 1 0
0 0 1 0
0 0 1 1
1 1 0 1
0 0 1 1
0 0 1 1
1 1 0 1
1 1 1 1
1 1 1 1
1 0 1 1
0 0 1 1"""), header=1)
print(df.dtypes)
indices = {cname: min_run_length(df[cname]) for cname in df.columns}
print(indices)
{
u'12': [(3, (3, 5)), (4, (11, 14))],
u'13': [(2, (4, 5)), (3, (11, 13))],
u'14': [(2, (1, 2)), (3, (5, 7)), (2, (9, 10)), (4, (12, 15))]
u'15': [(3, (2, 4)), (9, (7, 15))],
}
相关但不完全相同:这是一个非常聪明的解决方案!谢谢!