Python 在数据帧中合并行(在条件下)
我有一个数据框(df)(最初来自excel文件),前9行如下所示:Python 在数据帧中合并行(在条件下),python,pandas,data-cleaning,Python,Pandas,Data Cleaning,我有一个数据框(df)(最初来自excel文件),前9行如下所示: Control Recd_Date/Due_Date Action Signature/Requester 0 2000-1703 2000-01-31 00:00:00 OC/OER/OPA/PMS/ M WEBB 1 NaN 2000-02-29 00:00:00
Control Recd_Date/Due_Date Action Signature/Requester
0 2000-1703 2000-01-31 00:00:00 OC/OER/OPA/PMS/ M WEBB
1 NaN 2000-02-29 00:00:00 NaN DATA CORP
2 2000-1776 2000-01-02 00:00:00 OC/ORA/OE/DCP/ G KAN
3 NaN 2000-01-03 00:00:00 OC/ORA/ORO/PNC/ PALM POST
4 NaN NaN FDA/OGROP/ORA/SE-FO/FLA- NaN
5 NaN NaN DO/FLA-CB/ NaN
6 2000-1983 2000-02-02 00:00:00 FDA/OGROP/ORA/CE-FO/CHI- M EGAN
7 NaN 2000-02-03 00:00:00 DO/CHI-CB/ BERNSTEIN LIEBHARD &
8 NaN NaN NaN LONDON LLP
- 类型(df['Control'][1])=float李>
- 类型(df['Recd_Date/Due_Date'][1])=datetime.datetime李>
- 类型(df['Action_Office'][1])=浮动李>
- 类型(df['Signature/Requester'][1])=unicode
- Everytime pd.isnull(行['Control'])(这应该是唯一的条件)为true时,然后将此行与前一行(其'Control'值不为null)合并李>
- 对于“记录日期/到期日期”和“签名/请求者”,在两个值(从两个合并行)之间添加“,”(或“/”)(例如,“2000-01-31 00:00:002000-02-29 00:00:00”和“g KAN,PALM POST”)
- 对于“操作”,只需合并它们,而不添加任何标点符号(例如FDA/OGROP/ORA/CE-FO/CHI-DO/CHI-CB/)
for i, row in df.iterrows():
if pd.isnull(df.ix[i]['Control_#']):
df.ix[i-1]['Recd_Date/Due_Date'] = str(df.ix[i-1]['Recd_Date/Due_Date'])+'/'+str(df.ix[i]['Recd_Date/Due_Date'])
df.ix[i-1]['Subject'] = str(df.ix[i-1]['Subject'])+' '+str(df.ix[i]['Subject'])
if str(df.ix[i-1]['Action_Office'])[-1] == '-':
df.ix[i-1]['Action_Office'] = str(df.ix[i-1]['Action_Office'])+str(df.ix[i]['Action_Office'])
else:
df.ix[i-1]['Action_Office'] = str(df.ix[i-1]['Action_Office'])+','+str(df.ix[i]['Action_Office'])
if pd.isnull(df.ix[i-1]['Signature/Requester']):
df.ix[i-1]['Signature/Requester'] = str(df.ix[i-1]['Signature/Requester'])+str(df.ix[i]['Signature/Requester'])
elif str(df.ix[i-1]['Signature/Requester'])[-1] == '&':
df.ix[i-1]['Signature/Requester'] = str(df.ix[i-1]['Signature/Requester'])+' '+str(df.ix[i]['Signature/Requester'])
else:
df.ix[i-1]['Signature/Requester'] = str(df.ix[i-1]['Signature/Requester'])+','+str(df.ix[i]['Signature/Requester'])
df.drop(df.index[i])
为什么滴水不起作用?我正在尝试删除当前行(如果其['Control##']为null),以便可以将下一行(其['Control##']为null)以迭代方式添加到上一行(其['Control##']不为null)
非常感谢 我认为您需要将行分组在一起,然后将列值合并在一起。棘手的部分是找到一种方法,以您想要的方式将行组合在一起。这是我的解决方案 1) 将行分组在一起:静态变量 因为您的组依赖于行中的序列,所以我在方法中使用了一个静态变量来将每一行标记为一个特定的组
def rolling_group(val):
if pd.notnull(val): rolling_group.group +=1 #pd.notnull is signal to switch group
return rolling_group.group
rolling_group.group = 0 #static variable
此方法沿控件系列应用,以将索引排序为组,然后用于拆分数据帧以允许合并行
#groups = df.groupby(df['Control'].apply(rolling_group),as_index=False)
这确实是唯一棘手的部分,在这之后,您可以通过对每个组应用一个函数来合并行,从而提供所需的输出
完整解决方案代码
输出
Control Recd_Date/Due_Date \
0 2000-1703 2000-01-31 00:00:00,2000-02-29 00:00:00
1 2000-1776 2000-01-02 00:00:00,2000-01-03 00:00:00
2 2000-1983 2000-02-02 00:00:00,2000-02-03 00:00:00
Action \
0 OC/OER/OPA/PMS/
1 OC/ORA/OE/DCP/OC/ORA/ORO/PNC/FDA/OGROP/ORA/SE-...
2 FDA/OGROP/ORA/CE-FO/CHI-DO/CHI-CB/
Signature/Requester
0 M WEBB,DATA CORP
1 G KAN,PALM POST
2 M EGAN,BERNSTEIN LIEBHARD & LONDON LLP
你看过轮班方法了吗?谢谢你,伙计!我已经编辑了我的问题,有人能帮我吗?哦,非常感谢!!在我的原始数据框中,键入(df['Recd_Date/Due_Date'][i])=datetime.datetime。所以,当我在我的数据帧上运行代码时,我得到“TypeError:sequence item 1:expected string或Unicode,datetime.datetime found”。。。我将['Recd_Date/Due_Date']列转换为str作为'df['Recd_Date/Due_Date']=df['Recd_Date/Due_Date']。apply(str)'但仍然得到相同的错误。为什么?谢谢。我相信这是因为string.join(iterable)方法只接受带有字符串的iterables。我编辑了joinFunction方法,将其列表理解部分中的所有项转换为字符串。其他列也使用非字符串/unicode类型。还有一个技巧是,您可以使用astype转换列数据类型。我添加了一些额外的正则表达式来清理联接,您知道代码中发生了什么吗?我使用re.sub行在每个条件下将“-/”之类的内容更改为“-/”。代码应该与NAT一起工作。是否要发布数据帧的创建,以便我可以使用您正在使用的数据类型
df.to_dict('records')[:10]
Hi所以我只使用了你的数据类型,我发布的所有代码都正常工作。至于我是如何从列表中删除空值的,请看这段代码<代码>[str(each)for each in col if pd.notnull(each)]这是一个列表理解,最后有一个if语句删除空值。该代码返回列中的所有非空值,这些值在另一个列表中转换为字符串。
def rolling_group(val):
if pd.notnull(val): rolling_group.group +=1 #pd.notnull is signal to switch group
return rolling_group.group
rolling_group.group = 0 #static variable
def joinFunc(g,column):
col =g[column]
joiner = "/" if column == "Action" else ","
s = joiner.join([str(each) for each in col if pd.notnull(each)])
s = re.sub("(?<=&)"+joiner," ",s) #joiner = " "
s = re.sub("(?<=-)"+joiner,"",s) #joiner = ""
s = re.sub(joiner*2,joiner,s) #fixes double joiner condition
return s
if __name__ == "__main__":
df = """ Control Recd_Date/Due_Date Action Signature/Requester
0 2000-1703 2000-01-31 00:00:00 OC/OER/OPA/PMS/ M WEBB
1 NaN 2000-02-29 00:00:00 NaN DATA CORP
2 2000-1776 2000-01-02 00:00:00 OC/ORA/OE/DCP/ G KAN
3 NaN 2000-01-03 00:00:00 OC/ORA/ORO/PNC/ PALM POST
4 NaN NaN FDA/OGROP/ORA/SE-FO/FLA- NaN
5 NaN NaN DO/FLA-CB/ NaN
6 2000-1983 2000-02-02 00:00:00 FDA/OGROP/ORA/CE-FO/CHI- M EGAN
7 NaN 2000-02-03 00:00:00 DO/CHI-CB/ BERNSTEIN LIEBHARD &
8 NaN NaN NaN LONDON LLP"""
df = pd.read_csv(StringIO.StringIO(df),sep = "\s\s+",engine='python')
groups = df.groupby(df['Control'].apply(rolling_group),as_index=False)
groupFunct = lambda g: pd.Series([joinFunc(g,col) for col in g.columns],index=g.columns)
print groups.apply(groupFunct)
Control Recd_Date/Due_Date \
0 2000-1703 2000-01-31 00:00:00,2000-02-29 00:00:00
1 2000-1776 2000-01-02 00:00:00,2000-01-03 00:00:00
2 2000-1983 2000-02-02 00:00:00,2000-02-03 00:00:00
Action \
0 OC/OER/OPA/PMS/
1 OC/ORA/OE/DCP/OC/ORA/ORO/PNC/FDA/OGROP/ORA/SE-...
2 FDA/OGROP/ORA/CE-FO/CHI-DO/CHI-CB/
Signature/Requester
0 M WEBB,DATA CORP
1 G KAN,PALM POST
2 M EGAN,BERNSTEIN LIEBHARD & LONDON LLP