Python:如何让挂起的进程在concurrent.futures池中释放控制?

Python:如何让挂起的进程在concurrent.futures池中释放控制?,python,concurrency,multiprocessing,python-multiprocessing,concurrent.futures,Python,Concurrency,Multiprocessing,Python Multiprocessing,Concurrent.futures,我使用 concurrent.futuresProcessPoolExecutor方法,如下所示 with concurrent.futures.ProcessPoolExecutor(max_workers=workers) as executor: futures_ocr = ([ executor.submit( MyProcessor, folder ) for

我使用
concurrent.futuresProcessPoolExecutor
方法,如下所示

with concurrent.futures.ProcessPoolExecutor(max_workers=workers) as executor:
        futures_ocr = ([
            executor.submit(
                MyProcessor,
                folder
            ) for folder in sub_folders
        ])

        is_cancel = wait_for(futures_ocr)
        if is_cancel:
            print 'shutting down executor'
            executor.shutdown()

def wait_for(futures):
    """Handes the future tasks after completion"""

    cancelled = False

    try:
        for future in concurrent.futures.as_completed(futures, timeout=200):
            try:
                result = future.result()
                print 'successfully finished processing folder: ', result.source_folder_path

            except concurrent.futures.TimeoutError:
                print 'TimeoutError occured'


            except TypeError:
                print 'TypeError occured'



    except KeyboardInterrupt:
        print '****** cancelling... *******'
        cancelled = True
        for future in futures:
            future.cancel()

    return cancelled
在某些文件夹中,进程似乎被阻塞了很长时间,这不是因为代码中的某些错误,而是由于正在处理的文件的性质。所以,我想让这些类型的进程超时,以便在超过某个时间限制时返回。然后,池可以将该进程用于下一个可用任务

as_completed()
函数中添加超时会在完成时出错

Traceback (most recent call last):
  File "call_ocr.py", line 96, in <module>
    main()
  File "call_ocr.py", line 42, in main
    is_cancel = wait_for(futures_ocr)
  File "call_ocr.py", line 59, in wait_for
    for future in concurrent.futures.as_completed(futures, timeout=200):
  File "/Users/saurav/.pyenv/versions/ocr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 216, in as_completed
    len(pending), len(fs)))
concurrent.futures._base.TimeoutError: 3 (of 3) futures unfinished
回溯(最近一次呼叫最后一次):
文件“call_ocr.py”,第96行,在
main()
文件“call_ocr.py”,第42行,主目录
是否取消=等待(期货ocr)
文件“call_ocr.py”,第59行,等待
对于concurrent.futures.as_中的future(期货,超时=200):
文件“/Users/saurav/.pyenv/versions/ocr/lib/python2.7/site-packages/concurrent/futures/_-base.py”,第216行,填写完成
len(待定),len(fs)))
concurrent.futures.\u base.timeouter错误:3个(共3个)futures未完成

我在这里做错了什么?什么是使timedout进程停止并将进程放弃回进程池的最佳方法?

并发的
。futures
实现不支持这种用例

可以传递给函数和方法的
超时允许设置等待结果的时间,但对实际计算本身没有影响

库支持这样的用例

from concurrent.futures import TimeoutError
from pebble import ProcessPool

def function(n):
    return n

with ProcessPool() as pool:
    future = pool.schedule(function, args=[1], timeout=10)

    try:
        results = future.result()
    except TimeoutError as error:
        print("function took longer than %d seconds" % error.args[1])