Python 只要还有挂起的取消屏蔽任务,但不再存在,我如何运行asyncio循环?
我正在尝试向现有的asyncio循环中添加一些代码,以便在Ctrl-C上完全关闭。下面是它所做工作的抽象Python 只要还有挂起的取消屏蔽任务,但不再存在,我如何运行asyncio循环?,python,python-asyncio,Python,Python Asyncio,我正在尝试向现有的asyncio循环中添加一些代码,以便在Ctrl-C上完全关闭。下面是它所做工作的抽象 import asyncio, signal async def task1(): print("Starting simulated task1") await asyncio.sleep(5) print("Finished simulated task1") async def task2(): print("Starting simulated t
import asyncio, signal
async def task1():
print("Starting simulated task1")
await asyncio.sleep(5)
print("Finished simulated task1")
async def task2():
print("Starting simulated task2")
await asyncio.sleep(5)
print("Finished simulated task2")
async def tasks():
await task1()
await task2()
async def task_loop():
try:
while True:
await asyncio.shield(tasks())
await asyncio.sleep(60)
except asyncio.CancelledError:
print("Shutting down task loop")
raise
async def aiomain():
loop = asyncio.get_running_loop()
task = asyncio.Task(task_loop())
loop.add_signal_handler(signal.SIGINT, task.cancel)
await task
def main():
try:
asyncio.run(aiomain())
except asyncio.CancelledError:
pass
#def main():
# try:
# loop = asyncio.get_event_loop()
# loop.create_task(aiomain())
# loop.run_forever()
# except asyncio.CancelledError:
# pass
if __name__ == '__main__':
main()
在本例中,假设task1
和task2
的序列一旦启动就需要完成,否则某些工件将处于不一致的状态。(因此,asyncio.shield
围绕调用任务的包装器
)
对于上面的代码,如果我在脚本启动后不久中断脚本,并且脚本刚刚打印出来,则循环停止,而task2
永远不会启动。如果我尝试切换到已注释掉的main
版本,则该版本将永远不会退出,即使循环已正确取消,并且至少在几分钟内没有进一步的事件发生。它确实取得了一些进展,因为它至少完成了task1
和task2
的任何正在进行的序列
通过头脑风暴,我找到了一些可能的解决方案,尽管我仍然觉得我缺少了一些更简单的东西:
- 围绕
创建一个包装器,该包装器递增由asyncio.shield
对象同步的变量,运行屏蔽函数,然后递减该变量。然后,在asyncio.Condition
中的aiomain
处理程序中,等待变量达到零,然后重新引发异常。(在一个实现中,我可能会使用cancelederror
在\uuuu aexit\uuuu
取消错误逻辑上实现等待归零的功能,将该类的所有部分合并到一个类中。)
- 完全跳过使用
的取消机制,而是使用asyncio
或类似的机制来允许中断点或可中断休眠。虽然这看起来更具侵入性,要求我指定哪些点被认为是可中断的,而不是声明哪些序列需要屏蔽以避免取消asyncio.Event
这是一个非常好的问题。在解答问题的过程中,我学到了一些东西,所以我希望您仍在监视此线程 首先要调查的是,shield()方法是如何工作的?在这一点上,文档至少可以说是令人困惑的。在阅读test_tasks.py中的标准库测试代码之前,我无法理解它。以下是我的理解: 考虑以下代码片段:
async def coro_a():
await asyncio.sheild(task_b())
...
task_a = asyncio.create_task(coro_a())
task_a.cancel()
当执行task_a.cancel()语句时,task_a确实被取消。wait语句立即抛出CanceledError,而不等待task_b完成。但task_b继续运行。外部任务(a)停止,但内部任务(b)不停止
下面是您的程序的修改版本,说明了这一点。主要的变化是在CanceledError异常处理程序中插入一个等待,使程序的活动时间延长几秒钟。我在Windows上运行,这就是为什么我也稍微改变了你的信号处理程序,但这只是一个小问题。我还在打印报表中添加了时间戳
import asyncio
import signal
import time
async def task1():
print("Starting simulated task1", time.time())
await asyncio.sleep(5)
print("Finished simulated task1", time.time())
async def task2():
print("Starting simulated task2", time.time())
await asyncio.sleep(5)
print("Finished simulated task2", time.time())
async def tasks():
await task1()
await task2()
async def task_loop():
try:
while True:
await asyncio.shield(tasks())
await asyncio.sleep(60)
except asyncio.CancelledError:
print("Shutting down task loop", time.time())
raise
async def aiomain():
task = asyncio.create_task(task_loop())
KillNicely(task)
try:
await task
except asyncio.CancelledError:
print("Caught CancelledError", time.time())
await asyncio.sleep(5.0)
raise
class KillNicely:
def __init__(self, cancel_me):
self.cancel_me = cancel_me
self.old_sigint = signal.signal(signal.SIGINT,
self.trap_control_c)
def trap_control_c(self, signum, stack):
if signum != signal.SIGINT:
self.old_sigint(signum, stack)
else:
print("Got Control-C", time.time())
print(self.cancel_me.cancel())
def main():
try:
asyncio.run(aiomain())
except asyncio.CancelledError:
print("Program exit, cancelled", time.time())
# Output when ctrlC is struck during task1
#
# Starting simulated task1 1590871747.8977509
# Got Control-C 1590871750.8385916
# True
# Shutting down task loop 1590871750.8425908
# Caught CancelledError 1590871750.8435903
# Finished simulated task1 1590871752.908434
# Starting simulated task2 1590871752.908434
# Program exit, cancelled 1590871755.8488846
if __name__ == '__main__':
main()
您可以看到您的程序无法工作,因为在task1和task2有机会完成之前,task_循环一取消,程序就退出了。他们一直都在那里(或者更确切地说,如果程序继续运行,他们会一直在那里)
这说明了shield()和cancel()是如何交互的,但实际上并不能解决您所说的问题。为此,我认为,您需要有一个可等待的对象,您可以使用它来保持程序的运行,直到重要任务完成。这个对象需要在顶层创建,并沿着堆栈传递到执行重要任务的位置。这是一个与您的程序类似的程序,但是按照您想要的方式执行
我做了三次运行:(1)任务1期间的control-C,(2)任务2期间的control-C,(3)两个任务完成后的control-C。在前两种情况下,程序一直持续到task2完成。在第三个案例中,它立即结束
import asyncio
import signal
import time
async def task1():
print("Starting simulated task1", time.time())
await asyncio.sleep(5)
print("Finished simulated task1", time.time())
async def task2():
print("Starting simulated task2", time.time())
await asyncio.sleep(5)
print("Finished simulated task2", time.time())
async def tasks(kwrap):
fut = asyncio.get_running_loop().create_future()
kwrap.awaitable = fut
await task1()
await task2()
fut.set_result(1)
async def task_loop(kwrap):
try:
while True:
await asyncio.shield(tasks(kwrap))
await asyncio.sleep(60)
except asyncio.CancelledError:
print("Shutting down task loop", time.time())
raise
async def aiomain():
kwrap = KillWrapper()
task = asyncio.create_task(task_loop(kwrap))
KillNicely(task)
try:
await task
except asyncio.CancelledError:
print("Caught CancelledError", time.time())
await kwrap.awaitable
raise
class KillNicely:
def __init__(self, cancel_me):
self.cancel_me = cancel_me
self.old_sigint = signal.signal(signal.SIGINT,
self.trap_control_c)
def trap_control_c(self, signum, stack):
if signum != signal.SIGINT:
self.old_sigint(signum, stack)
else:
print("Got Control-C", time.time())
print(self.cancel_me.cancel())
class KillWrapper:
def __init__(self):
self.awaitable = asyncio.get_running_loop().create_future()
self.awaitable.set_result(0)
def main():
try:
asyncio.run(aiomain())
except asyncio.CancelledError:
print("Program exit, cancelled", time.time())
# Run 1 Control-C during task1
# Starting simulated task1 1590872408.6737766
# Got Control-C 1590872410.7344952
# True
# Shutting down task loop 1590872410.7354996
# Caught CancelledError 1590872410.7354996
# Finished simulated task1 1590872413.6747622
# Starting simulated task2 1590872413.6747622
# Finished simulated task2 1590872418.6750958
# Program exit, cancelled 1590872418.6750958
#
# Run 1 Control-C during task2
# Starting simulated task1 1590872492.927735
# Finished simulated task1 1590872497.9280624
# Starting simulated task2 1590872497.9280624
# Got Control-C 1590872499.5973852
# True
# Shutting down task loop 1590872499.5983844
# Caught CancelledError 1590872499.5983844
# Finished simulated task2 1590872502.9274273
# Program exit, cancelled 1590872502.9287038
#
# Run 1 Control-C after task2 -> immediate exit
# Starting simulated task1 1590873694.2925708
# Finished simulated task1 1590873699.2928336
# Starting simulated task2 1590873699.2928336
# Finished simulated task2 1590873704.2938952
# Got Control-C 1590873706.0790765
# True
# Shutting down task loop 1590873706.0804725
# Caught CancelledError 1590873706.0804725
# Program exit, cancelled 1590873706.0814824
以下是我最终使用的:
import asyncio, signal
async def _shield_and_wait_body(coro, finish_event):
try:
await coro
finally:
finish_event.set()
async def shield_and_wait(coro):
finish_event = asyncio.Event()
task = asyncio.shield(_shield_and_wait_body(coro, finish_event))
try:
await task
except asyncio.CancelledError:
await finish_event.wait()
raise
def shield_and_wait_decorator(coro_fn):
return lambda *args, **kwargs: shield_and_wait(coro_fn(*args, **kwargs))
async def task1():
print("Starting simulated task1")
await asyncio.sleep(5)
print("Finished simulated task1")
async def task2():
print("Starting simulated task2")
await asyncio.sleep(5)
print("Finished simulated task2")
@shield_and_wait_decorator
async def tasks():
await task1()
await task2()
async def task_loop():
try:
while True:
# Alternative to applying @shield_and_wait_decorator to tasks()
#await shield_and_wait(tasks())
await tasks()
await asyncio.sleep(60)
except asyncio.CancelledError:
print("Shutting down task loop")
raise
def sigint_handler(task):
print("Cancelling task loop")
task.cancel()
async def aiomain():
loop = asyncio.get_running_loop()
task = asyncio.Task(task_loop())
loop.add_signal_handler(signal.SIGINT, sigint_handler, task)
await task
def main():
try:
asyncio.run(aiomain())
except asyncio.CancelledError:
pass
if __name__ == '__main__':
main()
与Paul Cornelius的回答类似,这会插入一个等待子任务完成的过程,然后才允许CanceledError
向上传播调用链。但是,它不需要在调用asyncio.shield
的位置以外的地方触摸代码
(在我的实际用例中,我有三个循环同时运行,使用asyncio.Lock
来确保一个任务或一系列任务在另一个任务开始之前完成。在锁上我还有一个asyncio.Condition
从一个协程到另一个协程进行通信。当我尝试等待aiomain
或main
对于要完成的所有屏蔽任务,我遇到了一个问题,取消的父任务释放了锁,然后屏蔽任务尝试使用该锁向条件变量发送信号,给出了一个错误。将获取和释放锁移动到屏蔽任务中也没有意义-这将导致任务B仍在运行顺序:屏蔽任务A启动,任务B的协程终止其计时器并阻止等待锁、Control+C。另一方面,通过将等待放置在屏蔽_和_wait
调用点,它巧妙地避免了过早释放锁。)
一个警告:似乎
shield\u和\u wait\u decorator
在类方法上不能正常工作。谢谢-我用类似的想法发布了我自己的答案,但有一些评论为什么要“在顶层”等待最终没有为我工作,所以我的解决方案将等待转移到了shield
的包装中。这是一个棘手的问题。您的解决方案看起来不错,我喜欢decorator的想法。我能看到的唯一显著区别是,在您的解决方案中,task\u循环直到tasks()才“知道”取消