Fatal编程技术网
  • r/
  • R-ff包：查找ffdf中最频繁的元素并删除位于其中的行

R-ff包：查找ffdf中最频繁的元素并删除位于其中的行

r dataframe

R-ff包：查找ffdf中最频繁的元素并删除位于其中的行,r,dataframe,ff,ffbase,R,Dataframe,Ff,Ffbase,我需要一个建议来查找ffdf中最频繁的元素，然后删除位于其中的行。 我决定尝试ff软件包，因为我正在处理非常大的数据，而base R的内存正在耗尽 下面是一个小例子： # create a base R Matrix > z<-matrix(c("a", "b", "a", "c", "b", "b", "c", "c", "b", "a"),nrow=5,ncol=2,byrow = TRUE) > z [,1] [,2] [1,] "a" "b"

我需要一个建议来查找ffdf中最频繁的元素，然后删除位于其中的行。 我决定尝试ff软件包，因为我正在处理非常大的数据，而base R的内存正在耗尽

下面是一个小例子：

 # create a base R Matrix

 > z<-matrix(c("a", "b", "a", "c", "b", "b", "c", "c", "b", "a"),nrow=5,ncol=2,byrow = TRUE)
 > z


     [,1] [,2]
 [1,] "a"  "b" 
 [2,] "a"  "c" 
 [3,] "b"  "b" 
 [4,] "c"  "c" 
 [5,] "b"  "a" 


 # convert z to ffdf

 > u=as.data.frame(z, stringsAsFactors=TRUE)
 > u=as.ffdf(u)
 > u

  ffdf data
   V1 V2
1  a  b
2  a  c
3  b  b
4  c  c
5  b  a
在BaseR中，我找到了使用“table”函数的方法

temp
require（ff）
Z

   V1 V2
1  a  c
2  c  c



  temp <- table(as.vector(z))  
  t1<-names(temp)[temp == max(temp)] 
  z1<- z[rowSums(z== t1[1]) == 0, ]    



require(ff)
z <- matrix(c("a","b","f","c","f","b","e","c","b","e"),nrow=5,ncol=2,byrow = TRUE)
u <- as.data.frame(z, stringsAsFactors=TRUE)
u <- as.ffdf(u)
u



require(ffbase)
require(plyr)
## Detect most frequent item (assuming the levels of all columns can be different)
columnfreqs <- lapply(colnames(u), FUN=function(column) table(u[[column]]))
columnfreqs <- lapply(columnfreqs, FUN=function(x) as.data.frame(t(as.matrix(x))))
itemfreqs <- colSums(do.call(rbind.fill, columnfreqs), na.rm=TRUE)
mostfrequent <- names(sort(itemfreqs, decreasing = TRUE))[1]

## Identify the lines where the most frequent item occurs in each row of the ffdf 
idx <- ffrowapply(
  EXPR = apply(u[i1:i2,], MARGIN=1, FUN=function(row) any(row %in% mostfrequent)), 
  X=u, 
  RETURN = TRUE, FF_RETURN = TRUE, RETCOL = NULL, VMODE = "logical")
idx <- ffwhich(idx, idx != TRUE) # remove it is in there + convert logicals to integers

## Remove them
u[idx, ]