Python:列出每行具有相同值的列
我有一个数据框,下面是:Python:列出每行具有相同值的列,python,pandas,dataframe,Python,Pandas,Dataframe,我有一个数据框,下面是: import pandas as pd import numpy as np raw_data = {'col1': ['a', 'b', 'c', 'd', 'e'], 'col2': [1, 2, 3, 4, np.nan], 'col3': ['aa','b','cc','d','ff'], 'col4': [4, 6, 3, 4, np.nan] } df = pd.DataFrame(raw_d
import pandas as pd
import numpy as np
raw_data = {'col1': ['a', 'b', 'c', 'd', 'e'],
'col2': [1, 2, 3, 4, np.nan],
'col3': ['aa','b','cc','d','ff'],
'col4': [4, 6, 3, 4, np.nan]
}
df = pd.DataFrame(raw_data, columns = ['col1','col2','col3','col4'])
col1 col2 col3 col4
0 a 1.0 aa 4.0
1 b 2.0 b 6.0
2 c 3.0 cc 3.0
3 d 4.0 d 4.0
4 e NaN ff NaN
我想为每一行查找具有相同值的所有列。
所以结果应该是这样的:
Row 1: col1 eq col3;
Row 2: col2 eq col4;
Row 3: col1 eq col3; col2 eq col4
Dataframe有string和num列,因此可能值得将所有内容转换为str。
应忽略NaN数据值,因为缺少很多=)
非常感谢这里有一个for-loop解决方案,您可以使用它。。。也许piRSquared可以想出一个更好的、纯粹的解决方案。这在紧要关头应该行得通
row_eqs = {}
# For each row
for idx in df.index:
# Make a set of all "column equivalencies" for each row
row_eqs[idx] = set()
for col in df.columns:
# Look at all of the other columns that aren't `col`
other_cols = [c for c in df.columns if c != col]
# Column value
col_row_value = df.loc[idx, col]
for c in other_cols:
# Other column row value
c_row_value = df.loc[idx, c]
if c_row_value == col_row_value:
# Just make your strings here since lists and sets aren't hashable
eq = ' eq '.join(sorted((c, col)))
row_eqs[idx].add(eq)
打印结果:
for idx in row_eqs:
if row_eqs[idx]:
print('Row %d: %s' % (idx, '; '.join(row_eqs[idx])))
Row 1: col1 eq col3
Row 2: col2 eq col4
Row 3: col1 eq col3; col2 eq col4
编辑:一种稍微快一点的方法,通过预先硬编码列组合对的总数:
column_combos = {combo for combo in itertools.combinations(df.columns, 2)}
for idx in df.index:
row_eqs[idx] = set()
for col1, col2 in column_combos:
col1_value = df.loc[idx, col1]
col2_value = df.loc[idx, col2]
if col1_value == col2_value:
eq = ' eq '.join(sorted((col1, col2)))
row_eqs[idx].add(eq)
我不知道您的数据有多大,但后一种解决方案比前一种解决方案快25%。假设我们有以下DF:
In [1]: from numpy import nan
...: from itertools import combinations
...: import pandas as pd
...:
...: df = pd.DataFrame(
...: {'col1': {0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'},
...: 'col2': {0: 1.0, 1: 2.0, 2: 3.0, 3: 4.0, 4: nan},
...: 'col3': {0: 'aa', 1: 'b', 2: 'cc', 3: 'd', 4: 'ff'},
...: 'col4': {0: 4.0, 1: 6.0, 2: 3.0, 3: 4.0, 4: nan},
...: 'col5': {0: nan, 1: 'b', 2: 'c', 3: nan, 4: 'e'}})
...:
In [2]: df
Out[2]:
col1 col2 col3 col4 col5
0 a 1.0 aa 4.0 NaN
1 b 2.0 b 6.0 b
2 c 3.0 cc 3.0 c
3 d 4.0 d 4.0 NaN
4 e NaN ff NaN e
让我们用相同数据类型的所有列组合生成一个查询:
In [3]: qry = \
...: (df.dtypes
...: .reset_index(name='type')
...: .groupby('type')['index']
...: .apply(lambda x:
...: '\n'.join(['{0[0]}_{0[1]} = ({0[0]} == {0[1]})'.format(tup, tup)
...: for tup in combinations(x, 2)]))
...: .str.cat(sep='\n')
...: )
In [5]: print(qry)
col2_col4 = (col2 == col4)
col1_col3 = (col1 == col3)
col1_col5 = (col1 == col5)
col3_col5 = (col3 == col5)
现在我们可以这样做:
In [6]: cols = df.columns.tolist()
In [7]: (df.eval(qry, inplace=False)
...: .drop(cols, 1)
...: .apply(lambda r: ';'.join(r.index[r].tolist()).replace('_',' == '), axis=1)
...: )
Out[7]:
0
1 col1 == col3;col1 == col5;col3 == col5
2 col2 == col4;col1 == col5
3 col2 == col4;col1 == col3
4 col1 == col5
dtype: object
说明:
In [9]: df.eval(qry, inplace=False).drop(cols, 1)
Out[9]:
col2_col4 col1_col3 col1_col5 col3_col5
0 False False False False
1 False True True True
2 True False True False
3 True True False False
4 False False True False
这是我想出的另一个答案。我不知道对于没有一列具有相等值的行输出什么,所以我只在输出中跳过该行。还添加了一行,其中许多列具有相同的值,以显示那里发生了什么
import pandas as pd
import numpy as np
raw_data = {'col1': ['a', 'b', 'c', 'd', 'e', 1],
'col2': [1, 2, 3, 4, np.nan, 1],
'col3': ['aa','b','cc','d','ff', 1],
'col4': [4, 6, 3, 4, np.nan, 1],
}
df = pd.DataFrame(raw_data, columns = ['col1','col2','col3','col4'])
for row in df.itertuples():
values = list(set(row)) # Get the unique values in the row
equal_columns = [] # Keep track of column names that are the same
for v in values:
# Column names that have this value
columns = [df.columns[i-1] for i, x in enumerate(row) if x == v]
if len(columns) > 1:
# If more than 1 column with this value, append to the list
equal_columns.append(' eq '.join(columns))
if len(equal_columns) > 0:
# We have at least 1 set of equal columns
equal_columns.sort() # So we always start printing in lexicographic order
print('Row {0}: {1};'.format(row.Index, '; '.join(equal_columns)))
给我输出
Row 1: col1 eq col3;
Row 2: col2 eq col4;
Row 3: col1 eq col3; col2 eq col4;
Row 5: col1 eq col2 eq col3 eq col4;
另一种有效的方法:
a=df.values
equality=(a[:,newaxis,:]==a[:,:,newaxis])
match = row,col1,col2 = np.triu(equality,1).nonzero()
匹配现在是:
(array([1, 2, 3, 3], dtype=int64),
array([0, 1, 0, 1], dtype=int64),
array([2, 3, 2, 3], dtype=int64))
然后是漂亮的印刷:
dfc=df.columns
for i,r in enumerate(row):
print( str(r),' : ',str(dfc[col1[i]]),'=',str(dfc[col2[i]]))
用于:
我支持@not_a_robot的评论。我无法理解您的请求和声明的结果。请重新发布我删除的评论:您能否澄清第3行(索引2)的col1
如何等于col3
?我只看到了col1的c
和col3的cc
,它们在技术上是不相等的(尽管c
是cc
的适当子集)。您所需输出中显示的索引似乎已关闭…我正在从0开始计算行数。我的缺点是,我应该用“索引”来代替。值“c”和“cc”不应该被视为相等。我可能会尝试将事物转换为集合,并使用谢谢!数据集不是很大(约700Mb)。但是,不知道每行有多少列具有相同的值。@piRSquared谢谢您的时间。
1 : col1 = col3
2 : col2 = col4
3 : col1 = col3
3 : col2 = col4