Python 进程池中进程之间共享的类属性和内存？

我有一个类

a

，它在启动时会更改一个可变的类属性

nums

当通过

=1

的进程池启动该类时，我注意到

nums

具有多个不同进程的值。这对我来说是一种不受欢迎的行为

我的问题是：

• 进程是否共享内存
• 我是否没有正确理解进程池的工作原理

EDIT：我猜池会对它启动的前一个进程（而不是原始进程）进行pickle，从而保存

nums

的值，对吗？如果是这样的话，我怎么能强迫它使用原始流程呢

下面是一个示例代码：

from multiprocessing import Pool


class A:
    nums = []

    def __init__(self, num=None):
        self.__class__.nums.append(num)  # I use 'self.__class__' for the sake of explicitly
        print(self.__class__.nums)
        assert len(self.__class__.nums) < 2  # checking that they don't share memory


if __name__ == '__main__':
    with Pool(maxtasksperchild=1) as pool:
        pool.map(A, range(99))  # the assert is being raised

---

共享很可能是通过带有class属性

nums

的映射类

A

实现的

类属性是类绑定的，因此属于类本身，在加载类时创建，并由所有实例共享。所有对象对类属性都具有相同的内存引用

与类属性不同，实例属性是实例绑定的，不由各种实例共享。每个实例都有自己的实例属性副本

请参见“类与实例”属性效果：

1。使用

nums

作为类属性

运行此脚本

>>> python class_num.py
nums: [0]
nums: [0, 1]
nums: [4]
nums: [4, 5]
nums: [8]
nums: [8, 9]
nums: [12]
nums: [12, 13]
nums: [16]
nums: [16, 17]
nums: [20]
nums: [20, 21]
nums: [24]
nums: [24, 25]
nums: [28]
nums: [28, 29]
nums: [32]
nums: [32, 33]
nums: [36]
nums: [36, 37]
nums: [40]
nums: [40, 41]
nums: [44]
nums: [44, 45]
nums: [48]
nums: [48, 49]
nums: [52]
nums: [52, 53]
nums: [56]
nums: [56, 57]
nums: [60]
nums: [60, 61]
nums: [64]
nums: [64, 65]
nums: [68]
nums: [68, 69]
nums: [72]
nums: [72, 73]
nums: [76]
nums: [76, 77]
nums: [80]
nums: [80, 81]
nums: [84]
nums: [84, 85]
nums: [88]
nums: [88, 89]
nums: [92]
nums: [92, 93]
nums: [96]
nums: [96, 97]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "class_num.py", line 12, in __init__
    assert len(self.__class__.nums) < 2
AssertionError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "class_num.py", line 18, in <module>
    pool.map(A, range(99))  # the assert is being raised
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
AssertionError

>>> python instance_num.py
nums: [0]
nums: [1]
nums: [2]
nums: [3]
nums: [4]
nums: [5]
nums: [6]
nums: [7]
nums: [8]
nums: [9]
nums: [10]
nums: [11]
nums: [12]
nums: [13]
nums: [14]
nums: [15]
nums: [16]
nums: [17]
nums: [18]
nums: [19]
nums: [20]
nums: [21]
nums: [22]
nums: [23]
nums: [24]
nums: [25]
nums: [26]
nums: [27]
nums: [28]
nums: [29]
nums: [30]
nums: [31]
nums: [32]
nums: [33]
nums: [34]
nums: [35]
nums: [36]
nums: [37]
nums: [38]
nums: [39]
nums: [40]
nums: [41]
nums: [42]
nums: [43]
nums: [44]
nums: [45]
nums: [46]
nums: [47]
nums: [48]
nums: [49]
nums: [50]
nums: [51]
nums: [52]
nums: [53]
nums: [54]
nums: [55]
nums: [56]
nums: [57]
nums: [58]
nums: [59]
nums: [60]
nums: [61]
nums: [62]
nums: [63]
nums: [64]
nums: [65]
nums: [66]
nums: [67]
nums: [68]
nums: [69]
nums: [70]
nums: [71]
nums: [72]
nums: [73]
nums: [74]
nums: [75]
nums: [76]
nums: [77]
nums: [78]
nums: [79]
nums: [80]
nums: [81]
nums: [82]
nums: [83]
nums: [84]
nums: [85]
nums: [86]
nums: [87]
nums: [88]
nums: [89]
nums: [90]
nums: [91]
nums: [92]
nums: [93]
nums: [94]
nums: [95]
nums: [96]
nums: [97]
nums: [98]

---

你的观察还有另一个原因。

nums

中的值不是来自其他进程，而是来自开始承载多个实例的相同的进程。发生这种情况是因为在

池.map

-调用中没有将

chunksize

设置为1。 在您的情况下，设置

maxtasksparchild=1

是不够的，因为一个任务仍然会占用整个iterable块

此方法将iterable拆分为若干块，并将其作为单独的任务提交给流程池。可以通过将chunksize设置为正整数来指定这些块的（近似）大小

---

我理解类属性和实例属性之间的区别，但是这并不能回答我的问题。我需要使用类属性，因为在我的原始代码（这里没有显示）中，每个进程都有几个实例。我的问题是关于多处理池的工作原理。哦，我明白了。您可以从

pool（）

获得一个工作进程池

map

允许您通过对iterable中的每个元素应用函数并返回结果来实现执行和数据并行。因此，您向它提供函数和参数，池中生成的每个子进程都将能够成功导入。

>>> python class_num.py
nums: [0]
nums: [0, 1]
nums: [4]
nums: [4, 5]
nums: [8]
nums: [8, 9]
nums: [12]
nums: [12, 13]
nums: [16]
nums: [16, 17]
nums: [20]
nums: [20, 21]
nums: [24]
nums: [24, 25]
nums: [28]
nums: [28, 29]
nums: [32]
nums: [32, 33]
nums: [36]
nums: [36, 37]
nums: [40]
nums: [40, 41]
nums: [44]
nums: [44, 45]
nums: [48]
nums: [48, 49]
nums: [52]
nums: [52, 53]
nums: [56]
nums: [56, 57]
nums: [60]
nums: [60, 61]
nums: [64]
nums: [64, 65]
nums: [68]
nums: [68, 69]
nums: [72]
nums: [72, 73]
nums: [76]
nums: [76, 77]
nums: [80]
nums: [80, 81]
nums: [84]
nums: [84, 85]
nums: [88]
nums: [88, 89]
nums: [92]
nums: [92, 93]
nums: [96]
nums: [96, 97]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "class_num.py", line 12, in __init__
    assert len(self.__class__.nums) < 2
AssertionError
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "class_num.py", line 18, in <module>
    pool.map(A, range(99))  # the assert is being raised
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
AssertionError

from multiprocessing import Pool


class A:

    def __init__(self, num=None):
        self.nums = []
        if num is not None:
            self.nums.append(num)
        print("nums:", self.nums)
        # checking that they don't share memory
        assert len(self.nums) < 2


if __name__ == '__main__':
    with Pool(maxtasksperchild=1) as pool:
        pool.map(A, range(99))  # the assert is being raised

>>> python instance_num.py
nums: [0]
nums: [1]
nums: [2]
nums: [3]
nums: [4]
nums: [5]
nums: [6]
nums: [7]
nums: [8]
nums: [9]
nums: [10]
nums: [11]
nums: [12]
nums: [13]
nums: [14]
nums: [15]
nums: [16]
nums: [17]
nums: [18]
nums: [19]
nums: [20]
nums: [21]
nums: [22]
nums: [23]
nums: [24]
nums: [25]
nums: [26]
nums: [27]
nums: [28]
nums: [29]
nums: [30]
nums: [31]
nums: [32]
nums: [33]
nums: [34]
nums: [35]
nums: [36]
nums: [37]
nums: [38]
nums: [39]
nums: [40]
nums: [41]
nums: [42]
nums: [43]
nums: [44]
nums: [45]
nums: [46]
nums: [47]
nums: [48]
nums: [49]
nums: [50]
nums: [51]
nums: [52]
nums: [53]
nums: [54]
nums: [55]
nums: [56]
nums: [57]
nums: [58]
nums: [59]
nums: [60]
nums: [61]
nums: [62]
nums: [63]
nums: [64]
nums: [65]
nums: [66]
nums: [67]
nums: [68]
nums: [69]
nums: [70]
nums: [71]
nums: [72]
nums: [73]
nums: [74]
nums: [75]
nums: [76]
nums: [77]
nums: [78]
nums: [79]
nums: [80]
nums: [81]
nums: [82]
nums: [83]
nums: [84]
nums: [85]
nums: [86]
nums: [87]
nums: [88]
nums: [89]
nums: [90]
nums: [91]
nums: [92]
nums: [93]
nums: [94]
nums: [95]
nums: [96]
nums: [97]
nums: [98]