Python 3.x Python-如何在类实例中使用多处理锁?
我正在Windows上使用Python 3.7 我想做的是: -锁定一个类实例的方法,当另一个进程获得该锁时 尝试: 我已经成功地完成了,但是我不希望在这里为锁使用全局变量,而是完全在类内部的变量Python 3.x Python-如何在类实例中使用多处理锁?,python-3.x,locking,python-multiprocessing,python-3.7,Python 3.x,Locking,Python Multiprocessing,Python 3.7,我正在Windows上使用Python 3.7 我想做的是: -锁定一个类实例的方法,当另一个进程获得该锁时 尝试: 我已经成功地完成了,但是我不希望在这里为锁使用全局变量,而是完全在类内部的变量 from multiprocessing import Lock, freeze_support,Pool from time import sleep def do_work(name): print(name+' waiting for lock to work...',end='')
from multiprocessing import Lock, freeze_support,Pool
from time import sleep
def do_work(name):
print(name+' waiting for lock to work...',end='')
sleep(2)
with lock:
print('done!')
print(name+' doing work...',end='')
sleep(5)
print('done!')
def init(olock):
global lock
lock = olock
if __name__ == '__main__':
freeze_support()
args_list = [('a'),('b'),('c')]
lock=Lock()
p=Pool(8,initializer=init,initargs=(lock,))
p.map_async(do_work,args_list)
p.close()
p.join()
当最后一段代码运行时,由于锁的原因,需要17.3秒。如果没有锁,需要~7秒
class O():
def __init__(self):
self.lock=Lock()
def __getstate__(self): # used to remove multiprocess object(s) from class, so it can be pickled
self_dict=self.__dict__.copy()
del self_dict['lock']
return self_dict
def __setstate__(self,state): # used to remove multiprocess object(s) from class, so it can be pickled
self.__dict__.update(state)
def _do_work(self,name):
print(name+' waiting for lock to work...',end='')
sleep(2)
with self.lock:
print('done!')
print(name+' doing work...',end='')
sleep(5)
print('done!')
if __name__ == '__main__':
freeze_support()
c = O()
pool = Pool(8)
pool.apply_async(c._do_work,('a',))
pool.apply_async(c._do_work,('b',))
pool.apply_async(c._do_work,('c',))
pool.close()
pool.join()
我曾尝试在类中实现这一点,但锁不起任何作用,它总是在~7秒内运行
class O():
def __init__(self):
self.lock=Lock()
def __getstate__(self): # used to remove multiprocess object(s) from class, so it can be pickled
self_dict=self.__dict__.copy()
del self_dict['lock']
return self_dict
def __setstate__(self,state): # used to remove multiprocess object(s) from class, so it can be pickled
self.__dict__.update(state)
def _do_work(self,name):
print(name+' waiting for lock to work...',end='')
sleep(2)
with self.lock:
print('done!')
print(name+' doing work...',end='')
sleep(5)
print('done!')
if __name__ == '__main__':
freeze_support()
c = O()
pool = Pool(8)
pool.apply_async(c._do_work,('a',))
pool.apply_async(c._do_work,('b',))
pool.apply_async(c._do_work,('c',))
pool.close()
pool.join()
问题:
那么,当我调用一个通过多处理与资源异步交互的方法时,如何锁定这个类实例呢?
apply\u async
将pickle函数对象并通过队列发送到池工作进程,但由于c.\u do\u work
是一个绑定方法,该实例也将被pickle,这会导致一个错误。您可以将其包装在普通函数中:
c = O()
def w(*args):
return c._do_work(*args)
if __name__ == '__main__':
pool = Pool(1)
pool.apply_async(w, ('a',))
...
您应该删除\uuuuu setstate\uuuu
/\uuuuu getstate\uuuu