Python 如果任务失败,如何将任务放回队列?
我有一个脚本,看起来像这样:Python 如果任务失败,如何将任务放回队列?,python,multiprocessing,Python,Multiprocessing,我有一个脚本,看起来像这样: #!/usr/bin/env python # encoding: utf-8 import time, random, os, multiprocessing def main(): NPROCESSES = 5 pool = multiprocessing.Pool(processes=NPROCESSES) a = [1,2,3,4,5,6,7,8,9,0] for _ in pool.imap_unordered(do_
#!/usr/bin/env python
# encoding: utf-8
import time, random, os, multiprocessing
def main():
NPROCESSES = 5
pool = multiprocessing.Pool(processes=NPROCESSES)
a = [1,2,3,4,5,6,7,8,9,0]
for _ in pool.imap_unordered(do_task, a):
pass
def do_task(n):
try:
might_crash(n)
except Hell, e:
print e, " crashed."
def might_crash(n):
time.sleep(3*random.random())
if random.randrange( 3 ) == 0:
raise Hell(n)
print n
class Hell(Exception):
pass
if __name__=="__main__":
main()
此脚本通常会打印“a”中的值,但可能会随机引发异常
我希望捕获这些异常并将当前的do_task()放回队列中,以便稍后重试
如果当前任务失败,如何将其放回队列中 您可以从
do\u task
收集结果,检查哪些结果是Hell
的实例,将这些任务填入列表新任务
,然后循环直到没有新任务
:
import time
import random
import os
import multiprocessing as mp
def main():
NPROCESSES = 5
pool=mp.Pool(NPROCESSES)
a = [1,2,3,4,5,6,7,8,9,0]
new_tasks=a
while new_tasks:
a=new_tasks
new_tasks=[]
for result in pool.imap_unordered(do_task, a):
if isinstance(result,Hell):
new_tasks.append(result.args[0])
else:
print(result)
def do_task(n):
try:
result=might_crash(n)
except Hell as e:
print("{0} crashed.".format(e.args[0]))
result=e
return result
def might_crash(n):
time.sleep(3*random.random())
if random.randrange( 3 ) == 0:
raise Hell(n)
return '{0} done'.format(n)
class Hell(Exception):
pass
if __name__=="__main__":
main()
屈服
1 done
6 crashed.
4 done
7 crashed.
5 done
9 done
3 done
2 crashed.
8 done
0 crashed.
0 crashed.
2 done
7 crashed.
6 done
0 done
7 done