Python多处理管理器在启动时阻塞
python SyncManager在我的机器上初始化时似乎有问题 我有以下设置: 在我的主要启动程序中:Python多处理管理器在启动时阻塞,python,multiprocessing,deadlock,Python,Multiprocessing,Deadlock,python SyncManager在我的机器上初始化时似乎有问题 我有以下设置: 在我的主要启动程序中: if __name__ == '__main__': theManager = multiprocessManager.Manager() stopEvent = theManager.Event() # Make sure your daemon stops all threads in a clean way log = logging.getL
if __name__ == '__main__':
theManager = multiprocessManager.Manager()
stopEvent = theManager.Event() # Make sure your daemon stops all threads in a clean way
log = logging.getLogger('central-queue') # maybe move to mp.getlogger?
# do some log setup with file and stream handlers
log.info("Creating process safe Object to keep state of testbeds shared over all work processes")
testbedObjectsPool = theManager.TestbedInfo()
log.info("Wait a while before starting other processes to init the tbObjPool")
stopEvent.wait(100)
log.info("Starting process to watch the input dir")
jobFinderProcess = jobFinder.JobFinder(inputQueue, stopEvent)
jobFinderProcess.start()
在multiprocessManager.py中
from multiprocessing.managers import SyncManager
import testbedInfo
class MultiprocessManager(SyncManager):
pass
MultiprocessManager.register("TestbedInfo", testbedInfo.TestbedInfoPool, proxytype=None)
def Manager():
#print "MultiprocessManager.Manager"
m = MultiprocessManager()
m.start()
return m
在testbedInfo.py中
class TestbedInfoPool:
"""Gather all info from all testbeds into this class (ie a bunch of TestbedProperties objects)"""
def __init__(self):
# Specific to this class
self._initDatabase() # creates and configures self.db for us
self.reset()
blacklistfile = "%s/testbeds_blacklist_dispatcher.txt" % (os.getcwd())
self.populateTestbeds(blacklistfile) #Store the offline info into self.dictOfIdTestbedObjects Takes about 2 minutes
当它不工作时,我按下CTRL-C停止挂起过程,我在调试输出中得到以下结果
2017-02-08 09:35:10,141 - INFO - xMainProcessx =MainThread= Creating process safe Object to keep state of testbeds shared over all work processes
Traceback (most recent call last):
File "./dispatchToTestbed.py", line 117, in <module>
testbedObjectsPool = theManager.TestbedInfo()
File "/usr/global/lib/python2.7/multiprocessing/managers.py", line 641, in temp
token, exp = self._create(typeid, *args, **kwds)
File "/usr/global/lib/python2.7/multiprocessing/managers.py", line 541, in _create
id, exposed = dispatch(conn, None, 'create', (typeid,)+args, kwds)
File "/usr/global/lib/python2.7/multiprocessing/managers.py", line 76, in dispatch
kind, result = c.recv()
KeyboardInterrupt
有人知道如何进一步调试吗?或者我在这里遗漏了什么
我试图收集的更多信息:
主要流程是根据strace进行此操作:
write(7, "\0\0\0\t#WELCOME#", 13) = 13
write(7, "\0\0\0'\200\2(NU\6createq\1U\vTestbedInfoq"..., 43) = 43
read(7,
经理流程战略:
Process 26350 attached - interrupt to quit
[ Process PID=26350 runs in 32 bit mode. ]
accept(9,
lsof干管
dispatchT 26321 xxxxxx 7u unix 0xffff8808212260c0 0t0 637197167 socket
lsof经理
dispatchT 26350 xxxxxx 9u unix 0xffff88081eea5380 0t0 637197158 /tmp/pymp-v3RwL4/listener-ckeK4Y
那么这看起来像是主进程试图与manager进程通信,为TestbedInfo启动一个代理,但该进程根本没有收到它吗
Process 26350 attached - interrupt to quit
[ Process PID=26350 runs in 32 bit mode. ]
accept(9,
dispatchT 26321 xxxxxx 7u unix 0xffff8808212260c0 0t0 637197167 socket
dispatchT 26350 xxxxxx 9u unix 0xffff88081eea5380 0t0 637197158 /tmp/pymp-v3RwL4/listener-ckeK4Y