Python线程-动态更改并行线程的数量
我正在尝试concurrent.futures.ThreadPoolExecutor并行运行几个作业。它允许我在初始化时设置一次max_workers。但由于某些原因,我的要求是动态更改活动线程的数量。我找不到任何方法来做那件事。尝试动态重置“\u max\u workers”,但没有效果 这是密码Python线程-动态更改并行线程的数量,python,python-3.x,multithreading,python-multithreading,concurrent.futures,Python,Python 3.x,Multithreading,Python Multithreading,Concurrent.futures,我正在尝试concurrent.futures.ThreadPoolExecutor并行运行几个作业。它允许我在初始化时设置一次max_workers。但由于某些原因,我的要求是动态更改活动线程的数量。我找不到任何方法来做那件事。尝试动态重置“\u max\u workers”,但没有效果 这是密码 from concurrent.futures import ThreadPoolExecutor, as_completed import urllib.request from random i
from concurrent.futures import ThreadPoolExecutor, as_completed
import urllib.request
from random import randint
from time import sleep
URLS = ['http://www.foxnews.com/',
'http://www.cnn.com/',
'http://europe.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
# Retrieve a single page and report the url and contents
def load_url(url, timeout):
with urllib.request.urlopen(url, timeout=timeout) as conn:
return conn.read()
if __name__ == "__main__":
pool = ThreadPoolExecutor(1)
fs = []
for url in URLS:
fs.append(pool.submit(load_url, url, 60))
r = randint(1,5)
print ("Changing max_workers to: {}".format(r))
pool._max_workers = r
print ("Current queue size: {}. Number of threads in parallel: {}".format(pool._work_queue.qsize(), len(pool._threads)))
for f in as_completed(fs):
r = randint(1,5)
print ("Changing max_workers to: {}".format(r))
pool._max_workers = r
print ("Current queue size: {}. Number of threads in parallel: {}".format(pool._work_queue.qsize(), len(pool._threads)))
try:
data = f.result()
except Exception as exc:
print('%r generated an exception: %s' % (url, exc))
else:
print('Page is %d bytes' % (len(data)))
print ("done")
下面是它打印的内容:
Changing max_workers to: 3
Current queue size: 4. Number of threads in parallel: 1
Changing max_workers to: 1
Current queue size: 3. Number of threads in parallel: 1
Page is 233315 bytes
Changing max_workers to: 4
Current queue size: 2. Number of threads in parallel: 1
Page is 1127911 bytes
Changing max_workers to: 2
Current queue size: 1. Number of threads in parallel: 1
Page is 2741410 bytes
Changing max_workers to: 4
Current queue size: 0. Number of threads in parallel: 1
Page is 296361 bytes
done
有没有一种方法可以更改max_worker,它将更改池中的活动线程。_threads?嗯,我不知道如何使用concurrent.futures.ThreadPoolExecutor来完成此操作,但仍然能够在运行时更改活动线程的数量,使用一些围绕线程的自定义逻辑
这是最后的工作代码-嗯,我不知道如何使用concurrent.futures.ThreadPoolExecutor完成这项工作,但仍然在运行时通过一些自定义线程逻辑更改了活动线程的数量 这是最后的工作代码-