从R或python为并行selenium测试运行yaml文件

从R或python为并行selenium测试运行yaml文件,python,r,docker,selenium-grid,rselenium,Python,R,Docker,Selenium Grid,Rselenium,我有一个简单的yaml文件: seleniumhub: image: selenium/hub ports: - 4444:4444 firefoxnode: image: selenium/node-firefox-debug ports: - 4577 links: - seleniumhub:hub chromenode: image: selenium/node-chrome-debug po

我有一个简单的yaml文件:

seleniumhub:
    image: selenium/hub
    ports:
      - 4444:4444

firefoxnode:
    image: selenium/node-firefox-debug
    ports:
      - 4577
    links:
      - seleniumhub:hub

chromenode:
    image: selenium/node-chrome-debug
    ports:
      - 4578
    links:
      - seleniumhub:hub
我在docker执行的:

docker-compose up -d
我有一个中心和两个节点在运行

现在我想并行运行两个非常简单的selenium命令(用RSelenium编写):

我想知道如何在Python或R中并行运行上述selenium命令。我试过几种方法,但都不管用。例如,在R中:

library(RSelenium)
remDr <- remoteDriver(remoteServerAddr = "192.168.99.100", port = 4444L)
remDr$open()
remDr$navigate("http://www.r-project.org")
remDr$screenshot(display = TRUE)
库(RSelenium)
remDr这是重复的

您可以使用上面答案中的代码来执行并行执行

library(RSelenium)
library(rvest)
library(magrittr)
library(foreach)
library(doParallel)

URLsPar <- c("http://www.bbc.com/", "http://www.cnn.com", "http://www.google.com",
             "http://www.yahoo.com", "http://www.twitter.com")
appHTML <- c()

(cl <- (detectCores() - 1) %>%  makeCluster) %>% registerDoParallel
# open a remoteDriver for each node on the cluster
clusterEvalQ(cl, {
  library(RSelenium)
  remDr <- remoteDriver$new(remoteServerAddr = ip, port = port)
  remDr$open()
})
myTitles <- c()
ws <- foreach(x = 1:length(URLsPar), .packages = c("rvest", "magrittr", "RSelenium"))  %dopar%  {
  remDr$navigate(URLsPar[x])
  remDr$getTitle()[[1]]
}

# close browser on each node
clusterEvalQ(cl, {
  remDr$close()
})

stopImplicitCluster()
库(RSelenium)
图书馆(rvest)
图书馆(magrittr)
图书馆(foreach)
图书馆(双平行)

URLsPar类似,例如node:Thank,但我想用R或Python来做,因为我不熟悉JavaScript、Java或C#(我已经找到了这3种语言的解决方案)。在这里定义parallel,你想在两个浏览器上运行相同的命令吗?您可以运行两个脚本,还是希望使用相同的脚本运行它们并使用线程?我希望从一个R或Python脚本运行尽可能多的浏览器(唯一的约束是RAM)。
remDr <- remoteDriver(remoteServerAddr = "192.168.99.100", port = 4577L)
remDr$open()
remDr$navigate("http://www.r-project.org")
remDr$screenshot(display = TRUE)
library(RSelenium)
library(rvest)
library(magrittr)
library(foreach)
library(doParallel)

URLsPar <- c("http://www.bbc.com/", "http://www.cnn.com", "http://www.google.com",
             "http://www.yahoo.com", "http://www.twitter.com")
appHTML <- c()

(cl <- (detectCores() - 1) %>%  makeCluster) %>% registerDoParallel
# open a remoteDriver for each node on the cluster
clusterEvalQ(cl, {
  library(RSelenium)
  remDr <- remoteDriver$new(remoteServerAddr = ip, port = port)
  remDr$open()
})
myTitles <- c()
ws <- foreach(x = 1:length(URLsPar), .packages = c("rvest", "magrittr", "RSelenium"))  %dopar%  {
  remDr$navigate(URLsPar[x])
  remDr$getTitle()[[1]]
}

# close browser on each node
clusterEvalQ(cl, {
  remDr$close()
})

stopImplicitCluster()