R 在一列中聚合数据并排除重复项

我试图将使用两个SQL查询的分析简化为一个。在这样做的过程中，我在单个SQL查询中将生物质数据与size类数据连接起来，从而创建了重复的数据。这是因为生物量已经是一个总和，是每个地点内分类单元名称的总生物量，即在我的新表中是一对多的值

为了摆脱2个SQL查询，我在两个data.table操作和最后的一个联接中完成了这项工作。另一种方法是进行计算并删除重复项两次。是否有一种方法可以通过使用data.table来避免这两种情况

示例数据 testdf的上述输出是我想要的输出。我的另一次尝试依赖于两个！重复呼叫。在我的头脑中，我希望能够在丰度计算中使用[，totbm:=sumbiorship，by=listuniquesite，spcode]，但这不起作用

testdf[, .(site = (site), biomass = biomass, totabn = sum(lnXabun), n = sum(abundance), minlngth = min(size_class), maxlngth = max(size_class)), by = list(spcode, taxa_name)][, totbm := sum(biomass), by = list(unique(site), spcode)]
Error in `[.data.table`(testdf[, .(site = (site), biomass = biomass, totabn = sum(lnXabun),  : The items in the 'by' or 'keyby' list are length (3,15). Each must be length 15; the same length as there are rows in x (after subsetting if i is provided).

替代方法：

alt <- bm[, .(site = site, taxa_name = taxa_name, biomass = biomass, totabn = sum(lnXabun), n = sum(abundance), minlngth = min(size_class), maxlngth = max(size_class)),
by = list(spcode)]
alt <- alt[!duplicated(alt, by = c("site", "spcode"))]
alt[, totbm := sum(biomass), by = list(spcode)]
alt[!duplicated(alt, by = "spcode"), c(1,3,5:9)]

---

除非我遗漏了什么，否则你会让自己有点复杂。 只需做一个明确的总结：

bm <- testdf[, .SD[1L], by = list(spcode, taxa_name, biomass, site) # distinct
             ][, .(totbm = sum(biomass)), by = "spcode"] # summary

---

正如我在评论中提到的，我不喜欢数据冗余的表格，但这里有一种解决问题的方法。基本上，不要使用某种“唯一”函数，而是按一组场地/分类单元名称输入索引号，以便可以将除第一个生物量值以外的所有值设置为0。那么，按spcode/taxa_名称计算的总和就可以了。当然，这假设一组地点/分类群名称值正好对应一个生物量值

testdf <- data.table(spcode = c(10008L, 10008L, 10002L, 10002L, 10006L, 10008L, 10008L, 10002L, 10002L, 10011L, 10002L, 10002L, 10006L, 10006L, 10006L), 
                         abundance = c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 1L, 4L, 2L), 
                         biomass = c(0.2, 0.2, 0.1, 0.1, 0.1, 0.3, 0.3, 0.1, 0.1, 0.5, 0.1, 0.1, 0.5, 0.5, 0.5), 
                         size_class = c(21L, 20L, 14L, 10L, 14L, 21L, 23L, 16L, 13L, 17L, 12L, 5L, 9L, 10L, 11L), 
                         site = c(907L, 907L, 907L, 907L, 907L, 914L, 914L, 914L, 914L, 914L, 910L, 910L, 910L, 910L, 910L), 
                         taxa_name = c("Hippoglossina stomata", "Hippoglossina stomata", "Symphurus atricaudus", "Symphurus atricaudus", "Microstomus pacificus", "Hippoglossina stomata", "Hippoglossina stomata", "Symphurus atricaudus", "Symphurus atricaudus", "Parophrys vetulus", "Symphurus atricaudus", "Symphurus atricaudus", "Microstomus pacificus", "Microstomus pacificus", "Microstomus pacificus"), 
                         lnXabun = c(21L, 20L, 14L, 10L, 14L, 21L, 23L, 16L, 26L, 17L, 12L, 5L, 9L, 40L, 22L))

testdf[, biomassIdx := 1:.N, by = c('site', 'taxa_name')]
testdf[biomassIdx > 1, biomass := 0]
testdf[, .(tatabn = sum(lnXabun), n = sum(abundance), minlngth = min(size_class), maxlngth = max(size_class) , bm = sum(biomass)),
        by = list(spcode, taxa_name)]

---

你的生物量计算首先添加了一个n，然后就没有用了？是的，很抱歉，这是一个遗漏的遗迹。我将删除它以避免混淆。从2到1个SQL查询是否节省了大量时间/处理？我建议不要构建一个有冗余的表，例如，在testdf中，taht taxa_name='Hippoglossina stomata'和site=907总是与biometry 0.2相匹配，特别是如果其他列（如lnXabun）没有相同的冗余模式，并且涉及聚合的话。您可能能够找到这个特定问题的解决方案，但今后它几乎肯定会绊倒您。@Andreas它节省了少量时间~30秒，因为这是根据年度数据完成的，但如果有任何风险，将其保存在单独的查询和计算中可以实现数据完整性。这无疑简化了生物量计算，但仍然需要连接。我试图弄清楚，如果没有联接或显式连接，您是否/如何可以执行这两个操作！重复。@Anonymouscoward哦，这确实让它变得更复杂。你也可以将前两行代码组合成testdf[rowidsite，taxa\u name>1L，biostatus:=0]

alt <- bm[, .(site = site, taxa_name = taxa_name, biomass = biomass, totabn = sum(lnXabun), n = sum(abundance), minlngth = min(size_class), maxlngth = max(size_class)),
by = list(spcode)]
alt <- alt[!duplicated(alt, by = c("site", "spcode"))]
alt[, totbm := sum(biomass), by = list(spcode)]
alt[!duplicated(alt, by = "spcode"), c(1,3,5:9)]

bm <- testdf[, .SD[1L], by = list(spcode, taxa_name, biomass, site) # distinct
             ][, .(totbm = sum(biomass)), by = "spcode"] # summary

testdf <- data.table(spcode = c(10008L, 10008L, 10002L, 10002L, 10006L, 10008L, 10008L, 10002L, 10002L, 10011L, 10002L, 10002L, 10006L, 10006L, 10006L), 
                         abundance = c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 1L, 4L, 2L), 
                         biomass = c(0.2, 0.2, 0.1, 0.1, 0.1, 0.3, 0.3, 0.1, 0.1, 0.5, 0.1, 0.1, 0.5, 0.5, 0.5), 
                         size_class = c(21L, 20L, 14L, 10L, 14L, 21L, 23L, 16L, 13L, 17L, 12L, 5L, 9L, 10L, 11L), 
                         site = c(907L, 907L, 907L, 907L, 907L, 914L, 914L, 914L, 914L, 914L, 910L, 910L, 910L, 910L, 910L), 
                         taxa_name = c("Hippoglossina stomata", "Hippoglossina stomata", "Symphurus atricaudus", "Symphurus atricaudus", "Microstomus pacificus", "Hippoglossina stomata", "Hippoglossina stomata", "Symphurus atricaudus", "Symphurus atricaudus", "Parophrys vetulus", "Symphurus atricaudus", "Symphurus atricaudus", "Microstomus pacificus", "Microstomus pacificus", "Microstomus pacificus"), 
                         lnXabun = c(21L, 20L, 14L, 10L, 14L, 21L, 23L, 16L, 26L, 17L, 12L, 5L, 9L, 40L, 22L))

testdf[, biomassIdx := 1:.N, by = c('site', 'taxa_name')]
testdf[biomassIdx > 1, biomass := 0]
testdf[, .(tatabn = sum(lnXabun), n = sum(abundance), minlngth = min(size_class), maxlngth = max(size_class) , bm = sum(biomass)),
        by = list(spcode, taxa_name)]