Python 3.x multiprocessing.Pool.map函数不使用';在全局命名空间中看不到变量

Python 3.x multiprocessing.Pool.map函数不使用';在全局命名空间中看不到变量,python-3.x,multiprocessing,global,Python 3.x,Multiprocessing,Global,此玩具脚本失败: import numpy as np import multiprocessing as mp def myFunc1(x): return x+c if __name__ == '__main__': c = np.array([[1,1],[1,1]]) b = np.array([[[1,2],[3,4]],[[5,6],[7,8]]]) pool = mp.Pool(processes=1) result = pool.map

此玩具脚本失败:

import numpy as np
import multiprocessing as mp

def myFunc1(x):
    return x+c

if __name__ == '__main__':
    c = np.array([[1,1],[1,1]])
    b = np.array([[[1,2],[3,4]],[[5,6],[7,8]]])
    pool = mp.Pool(processes=1)
    result = pool.map(myFunc1,b)
    for i in range(len(result)):
        print(result[i])
错误地

RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\aldo.dottavio\anaconda3\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\aldo.dottavio\anaconda3\lib\multiprocessing\pool.py", line 44, in mapstar
    return list(map(*args))
  File "C:\Users\aldo.dottavio\Projects\THEIAA\2020-02 parallel processing\untitled0.py", line 12, in myFunc1
    return x+c
NameError: name 'c' is not defined
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "C:\Users\aldo.dottavio\Projects\THEIAA\2020-02 parallel processing\untitled0.py", line 19, in <module>
    result = pool.map(myFunc1,b)

  File "C:\Users\aldo.dottavio\anaconda3\lib\multiprocessing\pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()

  File "C:\Users\aldo.dottavio\anaconda3\lib\multiprocessing\pool.py", line 644, in get
    raise self._value

NameError: name 'c' is not defined
结果是,在myFunc1中没有定义c。我认为这是由map造成的,并且通过使用map将myFunc1应用于数据,myFunc1将失去对全局名称空间的访问

类似的玩具脚本成功:

import numpy as np

def myFunc1(x):
    return x+c

if __name__ == '__main__':
    c = np.array([[1,1],[1,1]])
    b = np.array([[[1,2],[3,4]],[[5,6],[7,8]]])
    result = myFunc1(b)
    for i in range(len(result)):
        print(result[i])
并输出结果

[[2 3]
 [4 5]]
[[6 7]
 [8 9]]
正如所料

以上只是说明问题;在实际脚本中,函数更复杂,必须在数千个矩阵上迭代

我需要全局名称空间中的变量是函数使用的参数;它们不会被任何进程更改,并且对于所有函数调用都是相同的-它们在脚本中此时是只读的

如何从map中的函数访问全局变量?我不想复制参数-参数只有~1MB的数据,但我可以在iTable中包含15000个左右的元素。

我使用starmap和itertools.product()解决了这个问题

在c周围使用itertools.product和[]可以将整个数组c作为一个元素iterable传递,而不是迭代NumPy数组的第一个索引

[[2 3]
 [4 5]]
[[6 7]
 [8 9]]
import numpy as np
import multiprocessing as mp
from itertools import product

def myFunc1(x,c):
    return x+c

if __name__ == '__main__':

    c = np.array([[1,1],[1,1]])
    b = np.array([[[1,2],[3,4]],[[5,6],[7,8]]])
    pool = mp.Pool(processes=1)
    result = pool.starmap(myFunc1,product(b,[c]))
    # result = myFunc1(b)
    for i in range(len(result)):
        print(result[i])