使用输入向量SparkR按多列分组

使用输入向量SparkR按多列分组,r,sparkr,R,Sparkr,我正在使用SparkR 2.1.0进行数据操作 我希望以编程方式按多个列进行分组。我知道如果我单独列出它们,或者从向量中引用它们的位置,我可以按多个列进行分组。。。但我希望能够将列列表作为向量传递(这样,函数会自动根据我传递的参数数量进行调整) 虚拟数据: cpny您可以使用do.call()并将列和数据帧放入列表中。以下是我的作品: cpny <- c("Fakeco1", "Fakeco2", "Fakeco3", "Fakeco4", "Fakeco5", "Fakeco6") st

我正在使用SparkR 2.1.0进行数据操作

我希望以编程方式按多个列进行分组。我知道如果我单独列出它们,或者从向量中引用它们的位置,我可以按多个列进行分组。。。但我希望能够将列列表作为向量传递(这样,函数会自动根据我传递的参数数量进行调整)

虚拟数据:
cpny您可以使用do.call()并将列和数据帧放入列表中。以下是我的作品:

cpny <- c("Fakeco1", "Fakeco2", "Fakeco3", "Fakeco4", "Fakeco5", "Fakeco6")
state <- c("CA", "NY", "WA", "CA", "CA", "NY")
public <- c("Y", "Y", "N", "N", "N", "N")
color <- c("White", "Red", "Green", "Green", "Green", "Red")
revs <- c(400, 200, 900, 500, 200, 120)
df <- data.frame(cpny, state, public, color, revs)
# Convert to SparkR dataframe
df_s <- as.DataFrame(df)  

group_vars <- c('state', 'public')


function_params <- list(df_s)
for (i in range(1:length(group_vars))) {
    function_params[[i+1]] <- group_vars[i]
}

summarized<- do.call(SparkR::groupBy, function_params) %>%  SparkR::summarize(sum_Revs = sum(df_s$revs))
SparkR::head(summarized)
cpny另一个选择(更简洁一点)是编写
summated%…
  df_grouped <- df_s %>%
  groupBy('state', 'public') %>%
  summarize(sum_Revs = sum(df_s$revs))
  group_vars <- c('state', 'public')

  df_grouped <- df_s %>%
  groupBy(group_vars[[1]], group_vars[[2]]) %>%
  summarize(sum_Revs = sum(df_s$revs))
  group_vars <- c('state', 'public')

  df_grouped <- df_s %>%
  groupBy(group_vars) %>%
  summarize(sum_Revs = sum(df_s$revs))
cpny <- c("Fakeco1", "Fakeco2", "Fakeco3", "Fakeco4", "Fakeco5", "Fakeco6")
state <- c("CA", "NY", "WA", "CA", "CA", "NY")
public <- c("Y", "Y", "N", "N", "N", "N")
color <- c("White", "Red", "Green", "Green", "Green", "Red")
revs <- c(400, 200, 900, 500, 200, 120)
df <- data.frame(cpny, state, public, color, revs)
# Convert to SparkR dataframe
df_s <- as.DataFrame(df)  

group_vars <- c('state', 'public')


function_params <- list(df_s)
for (i in range(1:length(group_vars))) {
    function_params[[i+1]] <- group_vars[i]
}

summarized<- do.call(SparkR::groupBy, function_params) %>%  SparkR::summarize(sum_Revs = sum(df_s$revs))
SparkR::head(summarized)