Warning: file_get_contents(/data/phpspider/zhask/data//catemap/6/multithreading/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 在远程服务器上设置并发Selenium webdriver调用时出现的问题_Python_Multithreading_Selenium_Selenium Webdriver - Fatal编程技术网

Python 在远程服务器上设置并发Selenium webdriver调用时出现的问题

Python 在远程服务器上设置并发Selenium webdriver调用时出现的问题,python,multithreading,selenium,selenium-webdriver,Python,Multithreading,Selenium,Selenium Webdriver,当我在个人计算机上运行以下代码时,它运行正常: from selenium import webdriver from concurrent.futures import ThreadPoolExecutor def open_driver(num): options = webdriver.firefox.options.Options() options.headless = True print("Starting") with webdriver.Fir

当我在个人计算机上运行以下代码时,它运行正常:

from selenium import webdriver
from concurrent.futures import ThreadPoolExecutor

def open_driver(num):
    options = webdriver.firefox.options.Options()
    options.headless = True
    print("Starting")
    with webdriver.Firefox() as driver:
        pass
    print("Ending")
    return num

if __name__ == '__main__':
    with ThreadPoolExecutor(6) as executor:
        results = executor.map(open_driver,list(range(6)))
    print(list(results))
输出:

StartingStarting

Starting
Starting
StartingStarting

Ending
Ending
Ending
Ending
Ending
Ending
[0, 1, 2, 3, 4, 5]
但是,当我尝试在远程DigitalOcean服务器(运行Ubuntu 18.04版)上运行相同的程序时,会产生以下错误:

Traceback (most recent call last):
  File "multi_drive.py", line 32, in <module>
    print(list(results))
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "multi_drive.py", line 24, in open_driver
    with webdriver.Firefox(options = options, executable_path = '/home/ubuntu/.local/bin/geckodriver') as browser:
  File "/home/ubuntu/testing/venv/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/home/ubuntu/testing/venv/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/ubuntu/testing/venv/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ubuntu/testing/venv/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/ubuntu/testing/venv/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process
程序按预期运行(减去并发):

当我将池计数设置为2时,程序也会运行而不会出现错误,因此可能会出现一定程度的并发性(对于没有webdriver调用的代码,max_workers=6可以正常运行)

max_workers=3似乎是程序开始失败的阈值

(不确定是否相关,但此处的“cat/proc/sys/kernel/threads max”生成7677)

这是服务器的限制吗?(软件或硬件?)如何检查?否则,这里到底发生了什么??任何帮助都将不胜感激。多谢各位

 with ThreadPoolExecutor(1) as executor:
Starting
Finally
Starting
Finally
Starting
Finally
Starting
Finally
Starting
Finally
Starting
Finally
[0, 1, 2, 3, 4, 5]