R数据表使用日期的组子集总和

R数据表使用日期的组子集总和,r,data.table,R,Data.table,我的数据集如下所示: library(data.table) dt1 <- data.table(urn = c(rep("a", 5), rep("b", 4)), amount = c(10, 12, 23, 15, 19, 42, 11, 5, 10), date = as.Date(c("2016-01-01", "2017-01-02", "2017-02-04",

我的数据集如下所示:

library(data.table)    
dt1 <- data.table(urn = c(rep("a", 5), rep("b", 4)),
                  amount = c(10, 12, 23, 15, 19, 42, 11, 5, 10),
                  date = as.Date(c("2016-01-01", "2017-01-02", "2017-02-04",
                                   "2017-04-19", "2018-02-11", "2016-02-14",
                                   "2017-05-06", "2017-05-12", "2017-12-12")))
dt1
#    urn amount       date
# 1:   a     10 2016-01-01
# 2:   a     12 2017-01-02
# 3:   a     23 2017-02-04
# 4:   a     15 2017-04-19
# 5:   a     19 2018-02-11
# 6:   b     42 2016-02-14
# 7:   b     11 2017-05-06
# 8:   b      5 2017-05-12
# 9:   b     10 2017-12-12
由于我的数据量很大,我最好寻找一个
数据表
解决方案,但如果它可能比一个有1200万条记录的表更有效,我也可以选择其他解决方案。

希望这能有所帮助

dt1[, summed12m := {
        date_diff <- date - dt1$date
        sum(dt1$amount[date_diff >= 0 & date_diff <= 365 & urn==dt1$urn])
      }, 
      by = list(date, urn)]
样本数据:

dt1 <- structure(list(urn = c("a", "a", "a", "a", "a", "b", "b", "b", 
"b"), amount = c(10, 12, 23, 15, 19, 42, 11, 5, 10), date = structure(c(16801, 
17168, 17201, 17275, 17573, 16845, 17292, 17298, 17512), class = "Date")), .Names = c("urn", 
"amount", "date"), row.names = c(NA, -9L), class = c("data.table", 
"data.frame"), .internal.selfref = <pointer: 0x00000000027b0788>)

dt1这是对
foverlaps
的呼喊。我第一次使用foverlaps,因此我确信这里的一些专家可以更好地使用该功能。下面是:

dt1[, date2 := date]
rng <- dt1[, .(urn, enddate=date, 
    startdate=as.Date(paste(year(date)-1, month(date), mday(date), sep="-")))]
setkey(rng, urn, startdate, enddate)
foverlaps(dt1, rng, by.x=c("urn","date","date2"), type="within")[,
    sum(amount), by=.(urn, enddate)]

#    urn    enddate V1
# 1:   a 2016-01-01 10
# 2:   a 2017-01-02 12
# 3:   a 2017-02-04 35
# 4:   a 2017-04-19 50
# 5:   a 2018-02-11 34
# 6:   b 2016-02-14 42
# 7:   b 2017-05-06 11
# 8:   b 2017-05-12 16
# 9:   b 2017-12-12 26
dt1[,date2:=日期]

rng作为
foverlaps()
的替代方案,这也可以通过在非等联接中聚合来解决:

lubridate
用于日期算术,以避免其中一个日期为2月29日时发生意外

基本部分是非等联接

其中最后一列被选中以在
dt1
中创建新的
summed12m

补充说明 OP询问了
V1
V2
V3
的来源

表达式
(urn,date,date%m-%months(12))
动态创建一个新的data.table。(
()
数据表
列表()
的缩写)。由于未指定列名,
data.table
创建默认列名
V1
V2
,等等

不那么草率,表达式可以用显式命名的列重新编写

dt1[.(urn = urn, end = date, start = date %m-% months(12)), 
    on = .(urn, date <= end, date >= start), 
    sum(amount), by = .EACHI]
dt1[(urn=urn,end=date,start=date%m-%months(12)),
on=(urn,日期=开始),
总额(金额),按=.EACHI]

5行
中的
34
是从哪里得到的?这是第
4行
5行
的累计总和,因为它们都在同一个12个月内。这是一个很好的答案,但在我的大型数据集上进行测试时,速度相当慢。到目前为止,我们的时间长达15分钟。考虑到数据量(虽然
foverlaps
仍然很好),我认为这个答案最适合我的需要,但我不明白的是
V1
V2
从何而来。我真的希望能够比现在更好地理解这一点。我已经添加了一个改进的解释。这澄清了问题。干杯
dt1[, date2 := date]
rng <- dt1[, .(urn, enddate=date, 
    startdate=as.Date(paste(year(date)-1, month(date), mday(date), sep="-")))]
setkey(rng, urn, startdate, enddate)
foverlaps(dt1, rng, by.x=c("urn","date","date2"), type="within")[,
    sum(amount), by=.(urn, enddate)]

#    urn    enddate V1
# 1:   a 2016-01-01 10
# 2:   a 2017-01-02 12
# 3:   a 2017-02-04 35
# 4:   a 2017-04-19 50
# 5:   a 2018-02-11 34
# 6:   b 2016-02-14 42
# 7:   b 2017-05-06 11
# 8:   b 2017-05-12 16
# 9:   b 2017-12-12 26
library(lubridate)
dt1[, summed12m := dt1[.(urn, date, date %m-% months(12)), 
                       on = .(urn = V1, date <= V2, date >= V3), 
                       sum(amount), by = .EACHI]$V1][]
   urn amount       date summed12m
1:   a     10 2016-01-01        10
2:   a     12 2017-01-02        12
3:   a     23 2017-02-04        35
4:   a     15 2017-04-19        50
5:   a     19 2018-02-11        34
6:   b     42 2016-02-14        42
7:   b     11 2017-05-06        11
8:   b      5 2017-05-12        16
9:   b     10 2017-12-12        26
dt1[.(urn, date, date %m-% months(12)), 
    on = .(urn = V1, date <= V2, date >= V3), 
    sum(amount), by = .EACHI]
   urn       date       date V1
1:   a 2016-01-01 2015-01-01 10
2:   a 2017-01-02 2016-01-02 12
3:   a 2017-02-04 2016-02-04 35
4:   a 2017-04-19 2016-04-19 50
5:   a 2018-02-11 2017-02-11 34
6:   b 2016-02-14 2015-02-14 42
7:   b 2017-05-06 2016-05-06 11
8:   b 2017-05-12 2016-05-12 16
9:   b 2017-12-12 2016-12-12 26
dt1[.(urn = urn, end = date, start = date %m-% months(12)), 
    on = .(urn, date <= end, date >= start), 
    sum(amount), by = .EACHI]