Python多处理帮助按条件退出
我对Python中的多处理技术非常感兴趣,但我对这个问题没有任何兴趣。基本上,我有一个运行起来很耗时的过程。我需要在1到100的范围内运行它,但我希望在满足我要查找的条件后中止所有进程。返回值==90的条件 这是一段非多进程代码。有谁能给我一个例子,说明他们如何将其转换为多进程函数,一旦满足“90”的条件,代码将退出所有进程Python多处理帮助按条件退出,python,multiprocessing,python-multithreading,multiprocess,Python,Multiprocessing,Python Multithreading,Multiprocess,我对Python中的多处理技术非常感兴趣,但我对这个问题没有任何兴趣。基本上,我有一个运行起来很耗时的过程。我需要在1到100的范围内运行它,但我希望在满足我要查找的条件后中止所有进程。返回值==90的条件 这是一段非多进程代码。有谁能给我一个例子,说明他们如何将其转换为多进程函数,一旦满足“90”的条件,代码将退出所有进程 def Addsomething(i): SumOfSomething = i + 1 return SumOfSomething def Run
def Addsomething(i):
SumOfSomething = i + 1
return SumOfSomething
def RunMyProcess():
for i in range(100):
Something = Addsomething(i)
print Something
return
if __name__ == "__main__":
RunMyProcess()
编辑:
我在测试第三个版本时遇到了这个错误。你知道这是什么原因吗
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 554, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 507, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Python27\lib\multiprocessing\pool.py", line 379, in _handle_results
cache[job]._set(i, obj)
File "C:\Python27\lib\multiprocessing\pool.py", line 527, in _set
self._callback(self._value)
File "N:\PV\_Proposals\2013\ESS - Clear Sky\01-CODE\MultiTest3.py", line 20, in check_result
pool.terminate()
File "C:\Python27\lib\multiprocessing\pool.py", line 423, in terminate
self._terminate()
File "C:\Python27\lib\multiprocessing\util.py", line 200, in __call__
res = self._callback(*self._args, **self._kwargs)
File "C:\Python27\lib\multiprocessing\pool.py", line 476, in _terminate_pool
result_handler.join(1e100)
File "C:\Python27\lib\threading.py", line 657, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
也许这就是你要找的?请记住,我是为Python 3编写的。上面的print语句是Python2,在这种情况下,需要注意的是使用xrange而不是range
from argparse import ArgumentParser
from random import random
from subprocess import Popen
from sys import exit
from time import sleep
def add_something(i):
# Sleep to simulate the long calculation
sleep(random() * 30)
return i + 1
def run_my_process():
# Start up all of the processes, pass i as command line argument
# since you have your function in the same file, we'll have to handle that
# inside 'main' below
processes = []
for i in range(100):
processes.append(Popen(['python', 'thisfile.py', str(i)]))
# Wait for your desired process result
# Might want to add a short sleep to the loop
done = False
while not done:
for proc in processes:
returncode = proc.poll()
if returncode == 90:
done = True
break
# Kill any process that are still running
for proc in processes:
if proc.returncode is None:
# Might run into a race condition here,
# so might want to wrap with try block
proc.kill()
if __name__ == '__main__':
# Look for optional i argument here
parser = ArgumentParser()
parser.add_argument('i', type=int, nargs='?')
i = parser.parse_args().i
# If there isn't an i, then run the whole thing
if i is None:
run_my_process()
else:
# Otherwise, run your expensive calculation and return the result
returncode = add_something(i)
print(returncode)
exit(returncode)
编辑:
下面是一个更简洁的版本,它使用多处理模块而不是子流程:
from random import random
from multiprocessing import Process
from sys import exit
from time import sleep
def add_something(i):
# Sleep to simulate the long calculation
sleep(random() * 30)
exitcode = i + 1
print(exitcode)
exit(exitcode)
def run_my_process():
# Start up all of the processes
processes = []
for i in range(100):
proc = Process(target=add_something, args=[i])
processes.append(proc)
proc.start()
# Wait for the desired process result
done = False
while not done:
for proc in processes:
if proc.exitcode == 90:
done = True
break
# Kill any processes that are still running
for proc in processes:
if proc.is_alive():
proc.terminate()
if __name__ == '__main__':
run_my_process()
编辑2:
这是最后一个版本,我认为比其他两个版本好得多:
from random import random
from multiprocessing import Pool
from time import sleep
def add_something(i):
# Sleep to simulate the long calculation
sleep(random() * 30)
return i + 1
def run_my_process():
# Create a process pool
pool = Pool(100)
# Callback function that checks results and kills the pool
def check_result(result):
print(result)
if result == 90:
pool.terminate()
# Start up all of the processes
for i in range(100):
pool.apply_async(add_something, args=[i], callback=check_result)
pool.close()
pool.join()
if __name__ == '__main__':
run_my_process()
为什么不在print Something行之后放一个空的return呢?是要运行单独的进程还是单独的线程?向Popen进程发送kill信号很容易,但对于线程,您需要手动创建和检查事件对象之类的内容。另外,您能否澄清一下,您是只想在后台执行RunMyProcess,还是想为i的每个值添加并行执行的内容?此外,请记住,线程的返回值与函数的返回值并不相同。返回值需要通过轮询或回调函数来访问。我希望运行单独的进程。我的实际代码每次迭代运行大约需要30秒。对于这个示例,“Addsomething”表示我的代码,它花费了很长时间,并且每次迭代都需要运行。我想并行运行每一个I,但一旦达到90,就终止所有进程。我安装了2.7,但运行时没有任何错误。它每次也会在90停止。我想我已经掌握了它是如何工作的。我需要花一些时间来处理它,使它与我的代码一起工作。如果遇到问题,我会通知您。是否必须使用:if name==“main”调用:运行我的\u进程()如果遇到的错误是“RuntimeEror:cannotjoin current thread”,则这是在2.7和3.2中修复的Python错误。有关详细信息,请参阅。当我在mac电脑上运行内置的2.7时,就会出现这个错误。但它与我安装的3.3版本运行良好,看起来我使用的是2.7.2。最新版本是2.7.6。我尝试更新,看看是否有效。是的,更新到2.7.6解决了这个问题。谢谢