Python 3.x 上下文管理器中的concurrent.futures.ThreadPoolExecutor超时无法正常工作

有人能帮我解释一下为什么在上下文管理器中使用超时时超时不能正常工作吗

它在不使用上下文管理器的情况下正常工作，它将在5s后引发TimeoutException，但使用上下文管理器时，它不会在5s后引发异常

无上下文管理器的实现

from concurrent import futures
MAX_WORKERS = 20

def download_one(c):
    import time
    time.sleep(100)


def download_many():
    executor = futures.ThreadPoolExecutor(MAX_WORKERS)
    res = executor.map(download_one, [1,2,3,4],timeout=5)
    print(list(res))

    return len(list(res))


download_many()

from concurrent import futures
MAX_WORKERS = 20

def download_one(c):
    import time
    time.sleep(100)


def download_many():
    with futures.ThreadPoolExecutor(MAX_WORKERS) as executor:
        res = executor.map(download_one, [1,2,3,4],timeout=5)
        print(list(res))

    return len(list(res))

download_many()

使用上下文管理器实现

from concurrent import futures
MAX_WORKERS = 20

def download_one(c):
    import time
    time.sleep(100)


def download_many():
    executor = futures.ThreadPoolExecutor(MAX_WORKERS)
    res = executor.map(download_one, [1,2,3,4],timeout=5)
    print(list(res))

    return len(list(res))


download_many()

from concurrent import futures
MAX_WORKERS = 20

def download_one(c):
    import time
    time.sleep(100)


def download_many():
    with futures.ThreadPoolExecutor(MAX_WORKERS) as executor:
        res = executor.map(download_one, [1,2,3,4],timeout=5)
        print(list(res))

    return len(list(res))

download_many()

---

超时在这两种情况下都正常工作。调用

列表（res）

后，将引发

并发.futures.\u base.TimeoutError

5秒（=指定的超时值）。但是，当使用上下文管理器（带有语句的

时，将调用上下文管理器的
\uuuuuuuuuuuu
方法。在这种情况下，它将等到所有线程都完成后再离开上下文并（重新）引发原始错误

这可以通过在正确的位置捕获并记录异常来证明：

import concurrent
import logging
import time
from concurrent import futures

MAX_WORKERS = 20


def download_one(c):
    logging.info('download_one(%s)' % str(c))
    time.sleep(10)  # Note: reduced to 10 seconds


def sub(executor):
    try:
        futures = executor.map(download_one, [1,2,3,4], timeout=5)
    except concurrent.futures._base.TimeoutError:
        logging.info('map timed out!')  # this is never logged
        raise

    try:
        results = list(futures)
    except concurrent.futures._base.TimeoutError:
        logging.info('list timed out!')  # here it happens!
        raise

    logging.info(results)
    logging.info('sub done')
    return len(result)


def download_many1():  # without context manager
    logging.info('download_many1')
    executor = futures.ThreadPoolExecutor(MAX_WORKERS)
    return sub(executor)


def download_many2():  # with context manager
    logging.info('download_many2')
    with futures.ThreadPoolExecutor(MAX_WORKERS) as executor:
        return sub(executor)


logging.basicConfig(level=logging.DEBUG, format='%(asctime)-15s   %(message)s')

logging.info('start 1')
try:
    download_many1()
except concurrent.futures._base.TimeoutError:
    logging.info('timeout!')
finally:
    logging.info('1 finished\n')

logging.info('start 2')
try:
    download_many2()
except concurrent.futures._base.TimeoutError:
    logging.info('timeout!')
finally:
    logging.info('2 finished\n')

这将产生：

2019-04-27 21:17:20,578   start 1
2019-04-27 21:17:20,578   download_many1
2019-04-27 21:17:20,578   download_one(1)
2019-04-27 21:17:20,578   download_one(2)
2019-04-27 21:17:20,578   download_one(3)
2019-04-27 21:17:20,578   download_one(4)
2019-04-27 21:17:25,593   list timed out!   # actual timeout after 5 seconds
2019-04-27 21:17:25,593   timeout!          # the timeout you see at the same time
2019-04-27 21:17:25,593   1 finished

2019-04-27 21:17:25,593   start 2
2019-04-27 21:17:25,593   download_many2
2019-04-27 21:17:25,593   download_one(1)
2019-04-27 21:17:25,593   download_one(2)
2019-04-27 21:17:25,593   download_one(3)
2019-04-27 21:17:25,593   download_one(4)
2019-04-27 21:17:30,610   list timed out!   # actual timeout after 5 seconds
2019-04-27 21:17:35,608   timeout!          # the timeout you see 5 seconds later!!
2019-04-27 21:17:35,608   2 finished

实际上，在这两种情况下，都可能会引发一个
并发.futures.\u base.TimeoutError
。但是，在第二个示例中，它是在所有线程完成后打印的。减少您的超时时间（或等待更长时间）以查看它。我已经阅读了您链接到的线程，但这与我的案例不相同。这两个示例都会引发TimeoutError，但我不明白为什么在所有线程完成后都会引发second。除了在上下文管理器中退出时调用excecutor.shutdown（wait=True）外，所有操作都是相同的。我认为这是相同的行为，但可能很难理解（多线程和异常有时可能很难理解）。这当然是一个有趣的问题！我想出了一个解释/说明这种行为的答案。我希望它能回答你的问题。