Python 熊猫在最近的日期出生
我有一个特定主题的报告数据框架。每个报告都有分数,受试者在某些日期有分数,但在其他日期没有分数。我想创建一个新的数据框,它只包含每个主题的最新分数。下面是一个MRE。原始数据帧如下所示:Python 熊猫在最近的日期出生,python,pandas,Python,Pandas,我有一个特定主题的报告数据框架。每个报告都有分数,受试者在某些日期有分数,但在其他日期没有分数。我想创建一个新的数据框,它只包含每个主题的最新分数。下面是一个MRE。原始数据帧如下所示: Subject Rpt_date Score_1 Score_2 0 L. Skywalker 2020-12-01 9.0 NaN 1 L. Skywalker 2020-12-06 NaN 8.0 2 L. Skywalker 2
Subject Rpt_date Score_1 Score_2
0 L. Skywalker 2020-12-01 9.0 NaN
1 L. Skywalker 2020-12-06 NaN 8.0
2 L. Skywalker 2021-01-11 7.0 NaN
3 H. Solo 2020-11-19 NaN 7.0
4 H. Solo 2020-12-15 NaN 5.0
5 H. Solo 2021-01-26 4.0 NaN
6 L. Organa 2020-11-20 6.0 NaN
7 L. Organa 2020-12-01 NaN 6.0
8 L. Organa 2020-12-19 NaN 7.0
9 D. Djarin 2020-12-10 NaN 10.0
10 D. Djarin 2020-12-12 10.0 NaN
11 D. Djarin 2021-01-03 NaN 10.0
Subject Score_1 Score_2
0 L. Skywalker 7.0 8.0
1 H. Solo 4.0 5.0
2 L. Organa 6.0 7.0
3 D. Djarin 10.0 10.0
所需的输出如下所示:
Subject Rpt_date Score_1 Score_2
0 L. Skywalker 2020-12-01 9.0 NaN
1 L. Skywalker 2020-12-06 NaN 8.0
2 L. Skywalker 2021-01-11 7.0 NaN
3 H. Solo 2020-11-19 NaN 7.0
4 H. Solo 2020-12-15 NaN 5.0
5 H. Solo 2021-01-26 4.0 NaN
6 L. Organa 2020-11-20 6.0 NaN
7 L. Organa 2020-12-01 NaN 6.0
8 L. Organa 2020-12-19 NaN 7.0
9 D. Djarin 2020-12-10 NaN 10.0
10 D. Djarin 2020-12-12 10.0 NaN
11 D. Djarin 2021-01-03 NaN 10.0
Subject Score_1 Score_2
0 L. Skywalker 7.0 8.0
1 H. Solo 4.0 5.0
2 L. Organa 6.0 7.0
3 D. Djarin 10.0 10.0
我的MRE在技术上是有效的,但看起来像是一个乱七八糟的东西,在一个大数据帧中会非常慢
import pandas as pd
import numpy as np
def merge_latest(df, col):
df_temp = df.dropna(subset=[col]).copy()
df_temp['Last_rpt'] = df_temp.groupby('Subject')['Rpt_date'].transform('max')
df_temp.drop(df_temp.loc[df_temp['Rpt_date'] != df_temp['Last_rpt']]
.index, inplace=True)
d = dict(zip(df_temp['Subject'], df_temp[c]))
return d
df_data = {'Subject':['L. Skywalker', 'L. Skywalker', 'L. Skywalker',
'H. Solo', 'H. Solo', 'H. Solo',
'L. Organa', 'L. Organa', 'L. Organa',
'D. Djarin', 'D. Djarin', 'D. Djarin'],
'Rpt_date':['12/1/2020', '12/6/2020', '1/11/2021',
'11/19/2020', '12/15/2020', '1/26/2021',
'11/20/2020', '12/1/2020', '12/19/2020',
'12/10/2020', '12/12/2020', '1/3/2021'],
'Score_1':[9, np.nan, 7,
np.nan, np.nan, 4,
6, np.nan, np.nan,
np.nan, 10, np.nan],
'Score_2':[np.nan, 8, np.nan,
7, 5, np.nan,
np.nan, 6, 7,
10, np.nan, 10]}
df = pd.DataFrame(data=df_data)
df['Rpt_date'] = pd.to_datetime(df['Rpt_date'])
print(df)
fin_df = pd.DataFrame()
fin_df['Subject'] = df['Subject'].unique()
for c in ['Score_1', 'Score_2']:
merge_dict = merge_latest(df, c)
fin_df[c] = fin_df['Subject'].map(merge_dict)
print(fin_df)
首先使用报告日期排序,然后使用。这个解决方案是矢量化的
df['Rpt_date'] = pd.to_datetime(df['Rpt_date'])
fin_df = df.sort_values('Rpt_date').groupby('Subject', as_index=False).last()
输出
Subject Rpt_date Score_1 Score_2
0 D. Djarin 2021-01-03 10.0 10.0
1 H. Solo 2021-01-26 4.0 5.0
2 L. Organa 2020-12-19 6.0 7.0
3 L. Skywalker 2021-01-11 7.0 8.0
这太完美了。仅供使用此答案的任何人参考,我碰巧已经按日期对数据进行了排序,但是@Vishnudev正确地建议添加
排序\u值
,以确保.last()
采用所需的值。