Python 如何创建一个dataframe列,该列与pandas中的多个列组合
我有一些数据可以追踪公司名称随时间的变化。但是,我不想让每一个名称都在一行上更改,而是希望将它们连接到一个字段中 输入数据可通过以下方式生成:Python 如何创建一个dataframe列,该列与pandas中的多个列组合,python,pandas,pyspark,Python,Pandas,Pyspark,我有一些数据可以追踪公司名称随时间的变化。但是,我不想让每一个名称都在一行上更改,而是希望将它们连接到一个字段中 输入数据可通过以下方式生成: #Import the modules: import pandas as pd import numpy as np #Create the empty data frame: df = pd.DataFrame(columns=['dt','old_name','new_name']) #Populate the data frame: df.l
#Import the modules:
import pandas as pd
import numpy as np
#Create the empty data frame:
df = pd.DataFrame(columns=['dt','old_name','new_name'])
#Populate the data frame:
df.loc[len(df)] = ['01/01/2001', 'AAA', 'BBB']
df.loc[len(df)] = ['02/02/2002', 'BBB', 'CCC']
df.loc[len(df)] = ['03/03/2003', 'CCC', 'DDD']
#View the output:
df
我希望输出的样子可以使用以下方法创建:
#Create the empty data frame:
end_df = pd.DataFrame(columns=['dt','name'])
#Populate:
end_df.loc[len(end_df)] = ['01/01/2001', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['02/02/2002', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['03/03/2003', 'AAA-BBB-CCC-DDD']
#View the output:
end_df
编辑:我正在Pyspark2中使用pandas数据帧运行此代码-以防对语法产生任何影响。
此外,我的数据集中有多组名称。我的意思是,有更多的名称更改组与第一个组无关,第一个组的名称需要连接
样本分组输入:
#Create the empty data frame:
df = pd.DataFrame(columns=['dt','old_name','new_name'])
#Populate the data frame:
df.loc[len(df)] = ['01/01/2001', 'AAA', 'BBB']
df.loc[len(df)] = ['02/02/2002', 'BBB', 'CCC']
df.loc[len(df)] = ['03/03/2003', 'CCC', 'DDD']
df.loc[len(df)] = ['02/01/2001', 'XXX', 'YYY']
df.loc[len(df)] = ['03/02/2002', 'YYY', 'ZZZ']
#Create the empty data frame:
end_df = pd.DataFrame(columns=['dt','name'])
#Populate:
end_df.loc[len(end_df)] = ['01/01/2001', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['02/02/2002', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['03/03/2003', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['02/01/2001', 'XXX-YYY-ZZZ']
end_df.loc[len(end_df)] = ['03/02/2002', 'XXX-YYY-ZZZ']
样本分组输出:
#Create the empty data frame:
df = pd.DataFrame(columns=['dt','old_name','new_name'])
#Populate the data frame:
df.loc[len(df)] = ['01/01/2001', 'AAA', 'BBB']
df.loc[len(df)] = ['02/02/2002', 'BBB', 'CCC']
df.loc[len(df)] = ['03/03/2003', 'CCC', 'DDD']
df.loc[len(df)] = ['02/01/2001', 'XXX', 'YYY']
df.loc[len(df)] = ['03/02/2002', 'YYY', 'ZZZ']
#Create the empty data frame:
end_df = pd.DataFrame(columns=['dt','name'])
#Populate:
end_df.loc[len(end_df)] = ['01/01/2001', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['02/02/2002', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['03/03/2003', 'AAA-BBB-CCC-DDD']
end_df.loc[len(end_df)] = ['02/01/2001', 'XXX-YYY-ZZZ']
end_df.loc[len(end_df)] = ['03/02/2002', 'XXX-YYY-ZZZ']
如果您需要进一步的澄清,请告诉我 创建了两个
dict
:old\u new\u dict
从旧名称遍历到新名称,以及old\u new\u dict\u rev
从新名称遍历到旧名称:
old_new_dict = {k:v for k,v in zip(df.old_name,df.new_name)}
old_new_dict_rev = {v:k for k,v in zip(df.old_name,df.new_name)}
函数find_tree
,沿两个方向进行遍历,并将它们连接起来以创建名称的完整路径
def find_tree(name):
left_list = []
right_list = []
name_l, name_r = name, name
while(name_l in old_new_dict_rev):
left_list.append(old_new_dict_rev[name_l])
name_l = old_new_dict_rev[name_l]
left_list.reverse()
while(name_r in old_new_dict):
right_list.append(old_new_dict[name_r])
name_r = old_new_dict[name_r]
return "-".join(left_list + [name] + right_list)
将完整路径添加为数据帧中的name
列df
:
df['name'] = df['old_name'].apply(lambda x: find_tree(x))
end_df = df.drop(['old_name','new_name'], axis = 1)
end_df
# dt name
#0 01/01/2001 AAA-BBB-CCC-DDD
#1 02/02/2002 AAA-BBB-CCC-DDD
#2 03/03/2003 AAA-BBB-CCC-DDD
#3 02/01/2001 XXX-YYY-ZZZ
#4 03/02/2002 XXX-YYY-ZZZ
您需要
np.flant和np.unique
import numpy as np
end_df = pd.DataFrame(columns=['dt','name'])
end_df['dt']=df['dt'].copy()
flat=df[df.columns[1:]].values.flatten()
end_df['name']='-'.join(np.unique(flat))
print(end_df)
dt name
0 01/01/2001 AAA-BBB-CCC-DDD
1 02/02/2002 AAA-BBB-CCC-DDD
2 03/03/2003 AAA-BBB-CCC-DDD
为什么要将其标记为pyspark?您显示的所有内容都是熊猫。很抱歉造成混淆-我在还没完成的时候无意中发布了这个问题。我现在将添加编辑。Hi pyd。谢谢你的回复。我刚刚编辑了我的问题,因为我忘记了在我的数据集中有多组名称更改。除此之外,您的解决方案非常有效!您知道如何对每个名称更改进行“分组”吗?