Python 合并数据框和数据透视创建新列

我有两个输入数据帧

df1（注意，此DF可能有更多的数据列）

和df2

          a    b    c
Sample               
1       0.2  0.4  0.3
2       0.5  0.7  0.2
3       0.4  0.1  0.9
4       0.4  0.2  0.3
5       0.6  0.2  0.4

我想把它们结合起来，这样我可以得到以下结果：

        one_a  one_b  one_c  two_a  two_b  two_c     Sex
Animal                                                  
A         0.2    0.4    0.3    0.5    0.7    0.2    male
B         0.4    0.1    0.9    NaN    NaN    NaN  female
C         0.4    0.2    0.3    NaN    NaN    NaN    male
D         0.6    0.2    0.4    NaN    NaN    NaN  female

这就是我做事的方式：

df2.reset_index(inplace = True)
df3 = pd.melt(df2, id_vars=['Sample'], value_vars=list(cols))
df4 = pd.merge(df3, df1, on='Sample')
df4['moo'] = df4['Group'] + '_' + df4['variable']
df5 = pd.pivot_table(df4, values='value', index='Animal', columns='moo')
df6 = df1.groupby('Animal').agg('first')
pd.concat([df5, df6], axis=1).drop('Sample',1).drop('Group',1)

---

这很好，但对于大型数据集来说可能会很慢。我想知道是否有熊猫专业人士看到了更好的（阅读速度更快，效率更高）？我刚接触熊猫，可以想象这里有一些我不知道的捷径。

这里有几个步骤。关键是为了创建像

one_a one_b….这样的列。。。。两个c

，我们需要在

示例

索引中添加

时间

列以构建多级索引，然后

取消堆栈

以获得所需的表单。然后，需要使用

Animal

索引上的

groupby

来聚合和减少

NaN

s的数量。剩下的只是对格式的一些操作

import pandas as pd

# your data
# ==============================
# set index
df1 = df1.set_index('Sample')

print(df1)

       Animal Time     Sex
Sample                    
1           A  one    male
2           A  two    male
3           B  one  female
4           C  one    male
5           D  one  female

print(df2)


          a    b    c
Sample               
1       0.2  0.4  0.3
2       0.5  0.7  0.2
3       0.4  0.1  0.9
4       0.4  0.2  0.3
5       0.6  0.2  0.4



# processing
# =============================
df = df1.join(df2)

df_temp = df.set_index(['Animal', 'Sex','Time'], append=True).unstack()

print(df_temp)


                        a         b         c     
Time                  one  two  one  two  one  two
Sample Animal Sex                                 
1      A      male    0.2  NaN  0.4  NaN  0.3  NaN
2      A      male    NaN  0.5  NaN  0.7  NaN  0.2
3      B      female  0.4  NaN  0.1  NaN  0.9  NaN
4      C      male    0.4  NaN  0.2  NaN  0.3  NaN
5      D      female  0.6  NaN  0.2  NaN  0.4  NaN

# rename the columns if you wish
df_temp.columns = ['{}_{}'.format(x, y) for x, y in zip(df_temp.columns.get_level_values(1), df_temp.columns.get_level_values(0))]

print(df_temp)

                      one_a  two_a  one_b  two_b  one_c  two_c
Sample Animal Sex                                             
1      A      male      0.2    NaN    0.4    NaN    0.3    NaN
2      A      male      NaN    0.5    NaN    0.7    NaN    0.2
3      B      female    0.4    NaN    0.1    NaN    0.9    NaN
4      C      male      0.4    NaN    0.2    NaN    0.3    NaN
5      D      female    0.6    NaN    0.2    NaN    0.4    NaN


result = df_temp.reset_index('Sex').groupby(level='Animal').agg(max).sort_index(axis=1)

print(result)

           Sex  one_a  one_b  one_c  two_a  two_b  two_c
Animal                                                  
A         male    0.2    0.4    0.3    0.5    0.7    0.2
B       female    0.4    0.1    0.9    NaN    NaN    NaN
C         male    0.4    0.2    0.3    NaN    NaN    NaN
D       female    0.6    0.2    0.4    NaN    NaN    NaN

---

看起来这是一个很好的开始！我觉得这并不能很好地概括；尤其是重命名步骤。我的df1可以有任意数量的列（编辑OP）。有没有关于如何进一步推广的建议？@Constantino我已经更新了我的帖子。请让我知道这是否适合你；另一个泛化可以来自df_temp=df.set_index（df1\u cols，append=True）

import pandas as pd

# your data
# ==============================
# set index
df1 = df1.set_index('Sample')

print(df1)

       Animal Time     Sex
Sample                    
1           A  one    male
2           A  two    male
3           B  one  female
4           C  one    male
5           D  one  female

print(df2)


          a    b    c
Sample               
1       0.2  0.4  0.3
2       0.5  0.7  0.2
3       0.4  0.1  0.9
4       0.4  0.2  0.3
5       0.6  0.2  0.4



# processing
# =============================
df = df1.join(df2)

df_temp = df.set_index(['Animal', 'Sex','Time'], append=True).unstack()

print(df_temp)


                        a         b         c     
Time                  one  two  one  two  one  two
Sample Animal Sex                                 
1      A      male    0.2  NaN  0.4  NaN  0.3  NaN
2      A      male    NaN  0.5  NaN  0.7  NaN  0.2
3      B      female  0.4  NaN  0.1  NaN  0.9  NaN
4      C      male    0.4  NaN  0.2  NaN  0.3  NaN
5      D      female  0.6  NaN  0.2  NaN  0.4  NaN

# rename the columns if you wish
df_temp.columns = ['{}_{}'.format(x, y) for x, y in zip(df_temp.columns.get_level_values(1), df_temp.columns.get_level_values(0))]

print(df_temp)

                      one_a  two_a  one_b  two_b  one_c  two_c
Sample Animal Sex                                             
1      A      male      0.2    NaN    0.4    NaN    0.3    NaN
2      A      male      NaN    0.5    NaN    0.7    NaN    0.2
3      B      female    0.4    NaN    0.1    NaN    0.9    NaN
4      C      male      0.4    NaN    0.2    NaN    0.3    NaN
5      D      female    0.6    NaN    0.2    NaN    0.4    NaN


result = df_temp.reset_index('Sex').groupby(level='Animal').agg(max).sort_index(axis=1)

print(result)

           Sex  one_a  one_b  one_c  two_a  two_b  two_c
Animal                                                  
A         male    0.2    0.4    0.3    0.5    0.7    0.2
B       female    0.4    0.1    0.9    NaN    NaN    NaN
C         male    0.4    0.2    0.3    NaN    NaN    NaN
D       female    0.6    0.2    0.4    NaN    NaN    NaN