Python 如何用另一个数据框中最近的日期填充一个数据框中的日期列

我有一个数据帧访问=

另一个数据帧测量=

我想根据人员id和可能的最近日期，用就诊表的就诊发生id填写测量表的就诊发生id

我已经写了一个代码，但它需要很多时间

测量有7*10^5行

注：访问开始日期和测量日期是对象类型

我的代码执行时间很长。您能帮助我降低时间复杂度或使用其他解决方案吗

编辑-添加数据帧构造函数

import numpy as np

measurement = {'measurement_date':['2017-09-04', '2018-04-24', '2018-05-22', '2019-02-02', 
                                   '2019-01-28', '2019-05-07', '2018-12-11','2017-04-28'],
        'person_id':[1, 2, 2, 1, 3, 1, 3, 3],'visit_occurrence_id':[np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan]}

visit = {'visit_occurrence_id':[1, 2, 3, 4, 5], 
         'visit_start_date':['2016-06-01', '2019-05-01', '2016-01-22', '2017-02-14', '2018-05-11'],
         'person_id':[1, 2, 1, 2, 3]}

# Create DataFrame
measurement = pd.DataFrame(measurement)
visit = pd.DataFrame(visit)

---

您可以执行以下操作：

df=pd.mergemeasurement[[person\u id，measurement\u date]]，访问，on=person\u id，how=internal df[dt_diff]=df[[visit_start_date，measurement_date]].applylambda x:absdatetime.datetime.strptimex[visit_start_date]，“%Y-%m-%d”。date-datetime.strptimex[measurement_date]，“%Y-%m-%d”。日期，轴=1 df=pd.mergedf，df.groupby[person\u id，measurement\u date][dt\u diff].min，on=[person\u id，dt\u diff，measurement\u date]，how=internal res=pd.mergemeasurement，df，on=[measurement\u date，person\u id]，后缀=[，\u 2][[measurement\u date，person\u id，visit\u-id\u 2]] 输出：

测量日期人员id访问事件id 2 0 2017-09-04 1 1 1 2018-04-24 2 2 2 2018-05-22 2 2 3 2019-02-02 1 1 4 2019-01-28 3 5 5 2019-05-07 1 1 6 2018-12-11 3 5 7 2017-04-28 3 5

---

您可以执行以下操作：

df=pd.mergemeasurement[[person\u id，measurement\u date]]，访问，on=person\u id，how=internal df[dt_diff]=df[[visit_start_date，measurement_date]].applylambda x:absdatetime.datetime.strptimex[visit_start_date]，“%Y-%m-%d”。date-datetime.strptimex[measurement_date]，“%Y-%m-%d”。日期，轴=1 df=pd.mergedf，df.groupby[person\u id，measurement\u date][dt\u diff].min，on=[person\u id，dt\u diff，measurement\u date]，how=internal res=pd.mergemeasurement，df，on=[measurement\u date，person\u id]，后缀=[，\u 2][[measurement\u date，person\u id，visit\u-id\u 2]] 输出：

测量日期人员id访问事件id 2 0 2017-09-04 1 1 1 2018-04-24 2 2 2 2018-05-22 2 2 3 2019-02-02 1 1 4 2019-01-28 3 5 5 2019-05-07 1 1 6 2018-12-11 3 5 7 2017-04-28 3 5

---

以下是我的想法：

# Get all visit start dates
df = measurement.drop('visit_occurrence_id', axis=1).merge(visit, on='person_id')
df['date_difference'] = abs(df.measurement_date - df.visit_start_date)
# Find the smallest visit start date for each person_id - measurement_date pair
df['smallest_difference'] = df.groupby(['person_id', 'measurement_date'])['date_difference'].transform(min)
df = df[df.date_difference == df.smallest_difference]
df = df[['measurement_date', 'person_id', 'visit_occurrence_id']]
# Fill in visit_occurrence_id from original dataframe
measurement.drop("visit_occurrence_id", axis=1).merge(
    df, on=["measurement_date", "person_id"]
)

这将产生：

|    | measurement_date   |   person_id |   visit_occurrence_id |
|---:|:-------------------|------------:|----------------------:|
|  0 | 2017-09-04         |           1 |                     1 |
|  1 | 2018-04-24         |           2 |                     2 |
|  2 | 2018-05-22         |           2 |                     2 |
|  3 | 2019-02-02         |           1 |                     1 |
|  4 | 2019-01-28         |           3 |                     5 |
|  5 | 2019-05-07         |           1 |                     1 |
|  6 | 2018-12-11         |           3 |                     5 |
|  7 | 2017-04-28         |           3 |                     5 |

---

我相信使用sklearn可能会有一种更简洁的方式来写这篇文章：

以下是我的想法：

# Get all visit start dates
df = measurement.drop('visit_occurrence_id', axis=1).merge(visit, on='person_id')
df['date_difference'] = abs(df.measurement_date - df.visit_start_date)
# Find the smallest visit start date for each person_id - measurement_date pair
df['smallest_difference'] = df.groupby(['person_id', 'measurement_date'])['date_difference'].transform(min)
df = df[df.date_difference == df.smallest_difference]
df = df[['measurement_date', 'person_id', 'visit_occurrence_id']]
# Fill in visit_occurrence_id from original dataframe
measurement.drop("visit_occurrence_id", axis=1).merge(
    df, on=["measurement_date", "person_id"]
)

这将产生：

|    | measurement_date   |   person_id |   visit_occurrence_id |
|---:|:-------------------|------------:|----------------------:|
|  0 | 2017-09-04         |           1 |                     1 |
|  1 | 2018-04-24         |           2 |                     2 |
|  2 | 2018-05-22         |           2 |                     2 |
|  3 | 2019-02-02         |           1 |                     1 |
|  4 | 2019-01-28         |           3 |                     5 |
|  5 | 2019-05-07         |           1 |                     1 |
|  6 | 2018-12-11         |           3 |                     5 |
|  7 | 2017-04-28         |           3 |                     5 |

---

我相信，使用sklearn可能有一种更简洁的方法来编写此文档：

如果您可以为我们可以复制和粘贴的数据帧发布构造函数，您将更有可能得到答案。稍后我会尝试一下，如果您可以为我们可以复制和粘贴的数据帧发布构造函数，您将更有可能得到答案。我以后再试试

|    | measurement_date   |   person_id |   visit_occurrence_id |
|---:|:-------------------|------------:|----------------------:|
|  0 | 2017-09-04         |           1 |                     1 |
|  1 | 2018-04-24         |           2 |                     2 |
|  2 | 2018-05-22         |           2 |                     2 |
|  3 | 2019-02-02         |           1 |                     1 |
|  4 | 2019-01-28         |           3 |                     5 |
|  5 | 2019-05-07         |           1 |                     1 |
|  6 | 2018-12-11         |           3 |                     5 |
|  7 | 2017-04-28         |           3 |                     5 |