将并行Foreach调用导出到环境中
我想将foreach的输出导出到一个环境中。我正在从雅虎财经获取时间序列数据将并行Foreach调用导出到环境中,r,foreach,quantmod,parallel.foreach,doparallel,R,Foreach,Quantmod,Parallel.foreach,Doparallel,我想将foreach的输出导出到一个环境中。我正在从雅虎财经获取时间序列数据 library(quantmod) library(foreach) library(parallel) library(doParallel) registerDoParallel(cores=2) hub = new.env() tickers = c("NKE", "AAPL", "MSFT", "TSLA", "MPC", "PEP", "GIS", "MA","V", "CAT", "KHC", "AMZN
library(quantmod)
library(foreach)
library(parallel)
library(doParallel)
registerDoParallel(cores=2)
hub = new.env()
tickers = c("NKE", "AAPL", "MSFT", "TSLA", "MPC", "PEP", "GIS", "MA","V", "CAT", "KHC", "AMZN", "NFLX", "GS", "MS", "BAC", "GE", "KO", "JPM", "AMAT", "ABT", "BIIB")
#I have tried 2 methods below.
#The first gives me a list of just the ticker names.
#The second puts the data into a list. I am looking for an enviornment
foreach(r = tickers, .packages = "quantmod") %dopar% lapply(r, getSymbols, env = hub)
enviro = foreach(r = tickers, .packages = "quantmod")%dopar% lapply(r, getSymbols, auto.assign = F)
class(enviro)
[1] "list"
环境应该是这样的(当我不在foreach循环中运行它时,它会工作)
这个问题非常不清楚,但从您的问题来看,您似乎正在尝试将输出组合到单个环境中,以获得更高的速度 现在你应该注意一些事情
quantmod::getSymbols
对每个调用都有相当大的开销。使用当前方法,您应该会看到性能下降,因为每个符号都调用了函数
优化
减少开销的一种方法是将每个计算分为块。foreach
包依赖于迭代器
包,它允许人们非常简单地将计算分成块
nworker = 2
registerDoParallel(cores = nworker)
tickers = matrix(c("NKE", "AAPL", "MSFT", "TSLA", "MPC", "PEP", "GIS", "MA","V", "CAT", "KHC", "AMZN", "NFLX", "GS", "MS", "BAC", "GE", "KO", "JPM", "AMAT", "ABT", "BIIB"), ncol = 1)
tickerIter <- iterators::iter(tickers, by = 'row', #I made a 1 column matrix, so i will iterate over each row.
chunksize = ceiling(length(tickers)/nworker) #Set chunk size, such that each worker gets 1 job.
)
nworker = 2
registerDoParallel(cores = nworker)
tickers = matrix(c("NKE", "AAPL", "MSFT", "TSLA", "MPC", "PEP", "GIS", "MA","V", "CAT", "KHC", "AMZN", "NFLX", "GS", "MS", "BAC", "GE", "KO", "JPM", "AMAT", "ABT", "BIIB"), ncol = 1)
tickerIter <- iterators::iter(tickers, by = 'row', #I made a 1 column matrix, so i will iterate over each row.
chunksize = ceiling(length(tickers)/nworker) #Set chunk size, such that each worker gets 1 job.
)
output <- foreach(r = tickerIter,
.combine = function(...){
c(...) #Combine all outputs into a list
},
.multicombine = TRUE,
.packages = "quantmod")%dopar% {
currenv <- environment()
getSymbols(r, currenv)
as.list(currenv)
}
#If you really want it in a specific environment, you could use: (Could also be used in .combine)
list2env(output, hub)