Pandas 使用特定的开始/结束日期以及groupby对数据帧重新采样
我有一些像这样的事务数据Pandas 使用特定的开始/结束日期以及groupby对数据帧重新采样,pandas,pandas-groupby,Pandas,Pandas Groupby,我有一些像这样的事务数据 import pandas as pd from io import StringIO from datetime import datetime from datetime import timedelta data = """\ cust_id,datetime,txn_type,txn_amt 100,2019-03-05 6:30,Credit,25000 100,2019-03-06 7:42,Debit,4000 100,2019-03-07 8:54,D
import pandas as pd
from io import StringIO
from datetime import datetime
from datetime import timedelta
data = """\
cust_id,datetime,txn_type,txn_amt
100,2019-03-05 6:30,Credit,25000
100,2019-03-06 7:42,Debit,4000
100,2019-03-07 8:54,Debit,1000
101,2019-03-05 5:32,Credit,25000
101,2019-03-06 7:13,Debit,5000
101,2019-03-06 8:54,Debit,2000
"""
df = pd.read_table(StringIO(data), sep=',')
df['datetime'] = pd.to_datetime(df['datetime'], format='%Y-%m-%d %H:%M:%S')
# use datetime as the dataframe index
df = df.set_index('datetime')
print(df)
cust_id txn_type txn_amt
datetime
2019-03-05 06:30:00 100 Credit 25000
2019-03-06 07:42:00 100 Debit 4000
2019-03-07 08:54:00 100 Debit 1000
2019-03-05 05:32:00 101 Credit 25000
2019-03-06 07:13:00 101 Debit 5000
2019-03-06 08:54:00 101 Debit 2000
我想在每日一级对数据进行重采样,对txn\u金额
进行汇总(求和)cust\u id
和txn\u类型
的每种组合。同时,我想将索引标准化为5天(目前数据只包含3天的数据)。本质上,这就是我想要制作的:
cust_id txn_type txn_amt
datetime
2019-03-03 100 Credit 0
2019-03-03 100 Debit 0
2019-03-03 101 Credit 0
2019-03-03 101 Debit 0
2019-03-04 100 Credit 0
2019-03-04 100 Debit 0
2019-03-04 101 Credit 0
2019-03-04 101 Debit 0
2019-03-05 100 Credit 25000
2019-03-05 100 Debit 0
2019-03-05 101 Credit 25000
2019-03-05 101 Debit 0
2019-03-06 100 Credit 0
2019-03-06 100 Debit 4000
2019-03-06 101 Credit 0
2019-03-06 101 Debit 7000 => (note: aggregated value)
2019-03-07 100 Credit 0
2019-03-07 100 Debit 1000
2019-03-07 101 Credit 0
2019-03-07 101 Debit 0
到目前为止,我已经尝试创建一个新的日期时间索引,并尝试重新采样,然后使用新创建的索引,如下所示:
# create a 5 day datetime index
end_dt = max(df.index).to_pydatetime().strftime('%Y-%m-%d')
start_dt = max(df.index) - timedelta(days=4)
start_dt = start_dt.to_pydatetime().strftime('%Y-%m-%d')
dt_index = pd.date_range(start=start_dt, end=end_dt, freq='1D', name='datetime')
然而,我不知道如何进行分组部分。无分组的重采样输出错误的结果:
# resample timeseries so that one row is 1 day's worth of txns
df2 = df.resample(rule='D').sum().reindex(dt_index).fillna(0)
print(df2)
cust_id txn_amt
datetime
2019-03-03 0.0 0.0
2019-03-04 0.0 0.0
2019-03-05 201.0 50000.0
2019-03-06 302.0 11000.0
2019-03-07 100.0 1000.0
那么,在重新采样时,如何合并
cust\u id
和tsn\u type
的分组?我已经看到了,但是op的数据结构不同。我正在使用reindex
这里,关键是要设置多个
索引
df.index=pd.to_datetime(df.index).date
df=df.groupby([df.index,df['txn_type'],df['cust_id']]).agg({'txn_amt':'sum'}).reset_index(level=[1,2])
drange=pd.date_range(end=df.index.max(),periods =5)
idx=pd.MultiIndex.from_product([drange,df.cust_id.unique(),df.txn_type.unique()])
Newdf=df.set_index(['cust_id','txn_type'],append=True).reindex(idx,fill_value=0).reset_index(level=[1,2])
Newdf
Out[749]:
level_1 level_2 txn_amt
2019-03-03 100 Credit 0
2019-03-03 100 Debit 0
2019-03-03 101 Credit 0
2019-03-03 101 Debit 0
2019-03-04 100 Credit 0
2019-03-04 100 Debit 0
2019-03-04 101 Credit 0
2019-03-04 101 Debit 0
2019-03-05 100 Credit 25000
2019-03-05 100 Debit 0
2019-03-05 101 Credit 25000
2019-03-05 101 Debit 0
2019-03-06 100 Credit 0
2019-03-06 100 Debit 4000
2019-03-06 101 Credit 0
2019-03-06 101 Debit 7000
2019-03-07 100 Credit 0
2019-03-07 100 Debit 1000
2019-03-07 101 Credit 0
2019-03-07 101 Debit 0
令人印象深刻的答案,你能在下面的行中给出:
idx=pd.MultiIndex.from_product([drange,df.cust_id.unique(),df.txn_type.unique()])
@Erfan通过列表产品创建多索引,:-)