Python 数据块索引
我想找到数据集中每个数据块的起始索引和结束索引。 数据如下:Python 数据块索引,python,pandas,dataframe,indexing,Python,Pandas,Dataframe,Indexing,我想找到数据集中每个数据块的起始索引和结束索引。 数据如下: index A wanted_column1 wanted_column2 2000/1/1 0 0 2000/1/2 1 2000/1/2 1 2000/1/3 1 1 2000/1/4 1 1 2000/1/5 0 0 2000/1/6 1
index A wanted_column1 wanted_column2
2000/1/1 0 0
2000/1/2 1 2000/1/2 1
2000/1/3 1 1
2000/1/4 1 1
2000/1/5 0 0
2000/1/6 1 2000/1/6 2
2000/1/7 1 2
2000/1/8 1 2
2000/1/9 0 0
如数据所示,index
和A
是给定的列,wanted\u column1
和wanted\u column2
是我想要得到的。
其思想是存在不同的连续数据块。我想要检索每个数据块的起始索引,并且我想要增加数据中有多少块的计数
我尝试使用shift(-1),但无法区分起始索引和结束索引之间的差异。这就是您需要的吗
df['change'] = df['A'].diff().eq(1)
df['wanted_column1'] = df[['index','change']].apply(lambda x: x[0] if x[1] else None, axis=1)
df['wanted_column2'] = df['change'].cumsum()
df['wanted_column2'] = df[['wanted_column2','A']].apply(lambda x: 0 if x[1]==0 else x[0], axis=1)
df.drop('change', axis=1, inplace=True)
这将产生:
index A wanted_column1 wanted_column2
0 2000/1/1 0 None 0
1 2000/1/2 1 2000/1/2 1
2 2000/1/3 1 None 1
3 2000/1/4 1 None 1
4 2000/1/5 0 None 0
5 2000/1/6 1 2000/1/6 2
6 2000/1/7 1 None 2
7 2000/1/8 1 None 2
8 2000/1/9 0 None 2
编辑:性能比较
的解决方案:gehbiszumeis
19.9ms
解决方案:my
4.07毫秒
df
,您可以在df['A']中找到索引0
。前面的索引是chunck的最后一个索引,是块的第一个索引之后的索引。稍后,计算找到的索引的数量以计算数据块的数量
import pandas as pd
# Read your data
df = pd.read_csv('my_txt.txt', sep=',')
df['wanted_column1'] = None # creating already dummy columns
df['wanted_column2'] = None
# Find indices after each index, where 'A' is not 1, except of it is the last value
# of the dataframe
first = [x + 1 for x in df[df['A'] != 1].index.values if x != len(df)-1]
# Find indices before each index, where 'A' is not 1, except of it is the first value
# of the dataframe
last = [x - 1 for x in df[df['A'] != 1].index.values if x != 0]
# Set the first indices of each chunk at its corresponding position in your dataframe
df.loc[first, 'wanted_column1'] = df.loc[first, 'index']
# You can set also the last indices of each chunk (you only mentioned this in the text,
# not in your expected-result-listed). Uncomment for last indices.
# df.loc[last, 'wanted_column1'] = df.loc[last, 'index']
# Count the number of chunks and fill it to wanted_column2
for i in df.index: df.loc[i, 'wanted_column2'] = sum(df.loc[:i, 'wanted_column1'].notna())
# Some polishing of the df after to match your expected result
df.loc[df['A'] != 1, 'wanted_column2'] = 0
这给
index A wanted_column1 wanted_column2
0 2000/1/1 0 None 0
1 2000/1/2 1 2000/1/2 1
2 2000/1/3 1 None 1
3 2000/1/4 1 None 1
4 2000/1/5 0 None 0
5 2000/1/6 1 2000/1/6 2
6 2000/1/7 1 None 2
7 2000/1/8 1 None 2
8 2000/1/9 0 None 0
并且适用于所有长度的
df
和数据中的块数嘿,有什么答案适合你的问题吗?