Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/batch-file/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 在少数情况下,将collections.namedtuple与ProcessPoolExecutor一起使用会遇到问题_Python_Multiprocessing_Deadlock_Namedtuple - Fatal编程技术网

Python 在少数情况下,将collections.namedtuple与ProcessPoolExecutor一起使用会遇到问题

Python 在少数情况下,将collections.namedtuple与ProcessPoolExecutor一起使用会遇到问题,python,multiprocessing,deadlock,namedtuple,Python,Multiprocessing,Deadlock,Namedtuple,这个卡在这里了 >>> import concurrent.futures >>> from collections import namedtuple >>> #1. Initialise namedtuple here >>> # tm = namedtuple("tm", ["pk"]) >>> class T: ... #2. Initiali

这个卡在这里了

>>> import concurrent.futures
>>> from collections import namedtuple
>>> #1. Initialise namedtuple here
>>> # tm = namedtuple("tm", ["pk"])  
>>> class T:  
...     #2. Initialise named tuple here
...     #tm = namedtuple("tm", ["pk"]) 
...     def __init__(self): 
...         #3: Initialise named tuple here
...         tm = namedtuple("tm", ["pk"])                       
...         self.x = {'key': [tm('value')]}  
...     def test1(self):  
...         with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor:  
...             results = executor.map(self.test, ["key"])  
...         return results  
...     def test(self, s): 
...         print(self.x[s])   
... 
>>> t = T().test1()
^CTraceback(最近一次通话最后一次):
文件“”,第1行,在
过程1:
test1中第10行的文件“”
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py”,第623行,在退出时__
自关闭(等待=真)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py”,第681行,处于关闭状态
self.\u队列\u管理\u线程.join()
join中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py”,第1044行
回溯(最近一次呼叫最后一次):
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py”,第297行,在bootstrap中
self.run()
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py”,第99行,运行中
自我目标(*自我参数,**自我参数)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py”,第233行,在进程中
call\u item=call\u queue.get(block=True)
get中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/queues.py”,第94行
res=self.\u recv\u bytes()
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py”,第216行,以recv_字节为单位
buf=自身接收字节(最大长度)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py”,第407行,以字节为单位
buf=自我记录(4)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py”,第379行,在
区块=读取(句柄,剩余)
键盘中断
self.\u等待状态锁定()
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py”,第1060行,在等待状态锁中
elif lock.acquire(块,超时):
键盘中断

如果我在类之外(在#1中)初始化命名的元组,在这种情况下,效果很好。如果我按照#2或#3进行初始化,请有人告诉我有什么问题吗?

您没有更改初始化命名双工的位置。您正在更改创建namedtuple类的位置

当您在具有
集合的模块“y”中创建名为“x”的namedtuple类时。namedtuple
,它的
\uuuu模块
设置为
'y'
,它的
\uuu qualname\uu
设置为
'x'
。酸洗和取消酸洗依赖于此类在这些属性所指示的
y.x
位置中的实际可用性,但在示例的案例2和案例3中,它不是


Python不能pickle namedtuple,这会中断进程间与工作进程的通信。在辅助进程中执行
self.test
依赖于在辅助进程中酸洗
self.test
并取消其拷贝,如果
self.x
是无法酸洗的类的实例,则不会发生这种情况。

感谢您的澄清。我现在明白问题所在了。当我尝试显式地pickle namedtuple时,会出现以下错误:_pickle.PicklingError:无法pickle:main上的attribute lookup tm失败
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
Process ForkProcess-1:
  File "<stdin>", line 10, in test1
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 623, in __exit__
    self.shutdown(wait=True)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py", line 681, in shutdown
    self._queue_management_thread.join()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1044, in join
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py", line 233, in _process_worker
    call_item = call_queue.get(block=True)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/queues.py", line 94, in get
    res = self._recv_bytes()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
    self._wait_for_tstate_lock()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1060, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt