Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/345.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
带有大型对象的Python多处理管道将挂起_Python_Dictionary_Multiprocessing_Pipe - Fatal编程技术网

带有大型对象的Python多处理管道将挂起

带有大型对象的Python多处理管道将挂起,python,dictionary,multiprocessing,pipe,Python,Dictionary,Multiprocessing,Pipe,下面我有一个简单的代码片段来演示这个问题 from multiprocessing import Pipe import time recv_end, send_end = Pipe(duplex=False) d = {'word'+str(elem): elem for elem in range(3000)} start_time = time.time() send_end.send(d) print('--- %s seconds ---' % (time.time()-start

下面我有一个简单的代码片段来演示这个问题

from multiprocessing import Pipe
import time

recv_end, send_end = Pipe(duplex=False)
d = {'word'+str(elem): elem for elem in range(3000)}

start_time = time.time()
send_end.send(d)
print('--- %s seconds ---' % (time.time()-start_time))
以上操作很好,速度足够快,没有问题。但如果我将大小设置为5000,它就会无限期地挂起:

from multiprocessing import Pipe
import time

recv_end, send_end = Pipe(duplex=False)
d = {'word'+str(elem): elem for elem in range(5000)}  # changed to 5000

start_time = time.time()
send_end.send(d)
print('--- %s seconds ---' % (time.time()-start_time))

管道是否有尺寸限制,或者这是一个不可复制的问题?如果你把尺寸再大一点怎么样?如果有大小限制,那么避免这个问题并通过管道发送大字典的最佳方法是什么?提前谢谢

出现此问题的原因是
Pipe.send()
是一个阻塞调用,它等待接收。阅读更多。 要使其工作,您可以创建如下代码所示的流程:

#!/usr/bin/env python
from multiprocessing import Pipe, Process
import time
import sys


def foo(conn):
    d = {'word'+str(elem): elem for elem in range(5000)}  # changed to 5000
    conn.send(d)
    conn.close()


recv_end, send_end = Pipe(duplex=False)
p = Process(target=foo, args=(send_end, ))
p.start()

start_time = time.time()
recv_end.recv()  # Try to comment and you will see that it waits for being received
p.join()
print('--- %s seconds ---' % (time.time()-start_time))

出现此问题的原因是
Pipe.send()
是一个阻塞调用,它等待接收。阅读更多。 要使其工作,您可以创建如下代码所示的流程:

#!/usr/bin/env python
from multiprocessing import Pipe, Process
import time
import sys


def foo(conn):
    d = {'word'+str(elem): elem for elem in range(5000)}  # changed to 5000
    conn.send(d)
    conn.close()


recv_end, send_end = Pipe(duplex=False)
p = Process(target=foo, args=(send_end, ))
p.start()

start_time = time.time()
recv_end.recv()  # Try to comment and you will see that it waits for being received
p.join()
print('--- %s seconds ---' % (time.time()-start_time))

谢谢你的回答!你能详细解释一下为什么创建这样的流程会让它工作吗?谢谢你的回答!你能详细说明一下为什么创建这样的流程会让它工作吗?