如何让多处理python应用程序干净地退出

当我运行一个使用多处理的python脚本时，我发现当它接收到Ctrl-C时，很难让它完全停止。必须多次按下Ctrl-C，屏幕上会出现各种错误消息

如何制作使用多处理和退出的python脚本 当它收到Ctrl-C时是否干净

以这个脚本为例

import numpy as np, time
from multiprocessing import Pool

def countconvolve(N):
    np.random.seed() # ensure seed is random

    count = 0
    iters = 1000000 # 1million

    l=12
    k=12
    l0=l+k-1

    for n in range(N):
        t = np.random.choice(np.array([-1,1], dtype=np.int8), size=l0 * iters)
        v = np.random.choice(np.array([-1,1], dtype=np.int8), size = l * iters)
        for i in xrange(iters):
            if (not np.convolve(v[(l*i):(l*(i+1))],
t[(l0*i):(l0*(i+1))], 'valid').any()):
                count += 1
    return count

if __name__ == '__main__':
    start = time.clock()

    num_processes = 8
    N = 13

    pool = Pool(processes=num_processes)
    res = pool.map(countconvolve, [N] * num_processes)
    print res, sum(res)

    print (time.clock() - start)

---

我相信中提到的try-catch可以被改编来涵盖它

如果您将pool.map调用包装在try-catch中，然后调用terminate和join，我认为这样就可以了

[编辑]

一些实验表明，沿着这些思路的一些方法效果良好：

from multiprocessing import Pool
import random
import time

def countconvolve(N):
    try:
        sleepTime = random.randint(0,5)
        time.sleep(sleepTime)
        count = sleepTime
    except KeyboardInterrupt as e:
        pass
    return count

if __name__ == '__main__':
    random.seed(0)
    start = time.clock()

    num_processes = 8
    N = 13

    pool = Pool(processes=num_processes)
    try:
        res = pool.map(countconvolve, [N] * num_processes)
        print res, sum(res)
        print (time.clock() - start)
    except KeyboardInterrupt as e:
        print 'Stopping..'

---

我简化了您的示例，以避免在我的机器上加载

numpy

进行测试，但关键部分是处理CTRL+C键按下的两个

try-except

调用。

Jon的解决方案可能更好，但这里使用的是信号处理程序。我在VBox虚拟机中进行了尝试，虽然速度非常慢，但效果很好。我希望这会有帮助

import numpy as np, time
from multiprocessing import Pool
import signal

# define pool as global
pool = None


def term_signal_handler(signum, frame):
    global pool

    print 'CTRL-C pressed'
    try:
        pool.close()
        pool.join()
    except AttributeError:
        print 'Pool has been already closed'


def countconvolve(N):
    np.random.seed() # ensure seed is random

    count = 0
    iters = 1000000 # 1million

    l=12
    k=12
    l0=l+k-1

    for n in range(N):
        t = np.random.choice(np.array([-1,1], dtype=np.int8), size=l0 * iters)
        v = np.random.choice(np.array([-1,1], dtype=np.int8), size = l * iters)
        for i in xrange(iters):
            if (not np.convolve(v[(l*i):(l*(i+1))],t[(l0*i):(l0*(i+1))], 'valid').any()):
                count += 1
    return count


if __name__ == '__main__':
    # Register the signal handler
    signal.signal(signal.SIGINT, term_signal_handler)

    start = time.clock()

    num_processes = 8
    N = 13

    pool = Pool(processes=num_processes)
    res = pool.map(countconvolve, [N] * num_processes)
    print res, sum(res)

    print (time.clock() - start)

---

我会尝试编写一个捕获ctrl-c点击并终止生成的进程的函数。我使用了一个信号处理程序来关闭线程，这对我来说是最干净的。+1：我喜欢这样一个事实，即业务逻辑不必处理ctrl+c篡改，但遗憾的是它需要一个全局变量来工作：-/