Python 3.x python多处理中如何处理imap_无序异常

我使用pool.imap_unordered对本地保存的不同txt文件应用函数

是否可以捕获异常并通过？ 如果我的代码遇到异常，它会阻塞整个循环

pool = Pool(processes=15)

results = {}
files = glob.glob('{}/10K_files/*.txt'.format(path_input))

for key, output in tqdm(pool.imap_unordered(process_file, files),total=len(files)):
    results[key] = output

我试过这样的方法：

pool = Pool(processes=15)

results = {}
files = glob.glob('{}/10K_files/*.txt'.format(path_input))

try:
    for key, output in tqdm(pool.imap_unordered(process_file, files), total=len(files)):
       results[key] = output
except:
    print("error")

但是我想从我开始的地方继续循环

---

谢谢

您可以在进程\文件中捕获异常并返回它。然后测试返回值是否为异常。以下是一个例子：

import os
import traceback
import multiprocessing as mp


def main():
    work_items = [i for i in range(20)]
    pool = mp.Pool()
    for result in pool.imap_unordered(process_file_exc, work_items):
        if isinstance(result, Exception):
            print("Got exception: {}".format(result))
        else:
            print("Got OK result: {}".format(result))


def process_file_exc(work_item):
    try:
        return process_file(work_item)
    except Exception as ex:
        return Exception("Err on item {}".format(work_item)
                         + os.linesep + traceback.format_exc())


def process_file(work_item):
    if work_item == 9:
        # this will raise ZeroDivisionError exception
        return work_item / 0
    return "{} * 2 == {}".format(work_item, work_item * 2)


if __name__ == '__main__':
    main()