Python多进程更改类实例已就位

我有一个类实例列表，我想并行调用同一个实例方法，使用pathos来pickle实例方法，真正的问题是当我想更改/添加一个属性到实例时，它不起作用，我认为这是因为pickle-to子进程是输入的深度副本。有人知道怎么解决这个问题吗？我不想改变编写实例方法的方式，比如返回一个值，然后将其组合在一起

from joblib import Parallel, delayed
import pathos.multiprocessing as mp
# import multiprocessing as mp 
import random
import os

pool = mp.Pool(mp.cpu_count())

class Person(object):
    def __init__(self, name):
        self.name = name

    def print_name(self, num):
        self.num = num
        print "worker {}, person name {}, received int {}".format(os.getpid(), self.name, self.num)


people = [Person('a'),
          Person('b'),
          Person('c'),
          Person('d'),
          Person('e'),
          Person('f'),
          Person('g'),
          Person('h')]


for i, per in enumerate(people):
    pool.apply_async(Person.print_name, (per, i) )

pool.close()
pool.join()
print 'their number'
for per in people:
    print per.num

这是输出，找不到num属性，我认为这是因为对这些副本进行了更改

In [1]: run delme.py
worker 13981, person name a, random int 0
worker 13982, person name b, random int 1
worker 13983, person name c, random int 2
worker 13984, person name d, random int 3
worker 13985, person name e, random int 4
worker 13986, person name f, random int 5
worker 13987, person name g, random int 6
worker 13988, person name h, random int 7
their number
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/chimerahomes/wenhoujx/brain_project/network_analysis/delme.py in <module>()
     39 print 'their number'
     40 for per in people:
---> 41     print per.num

AttributeError: 'Person' object has no attribute 'num'

这是输出：

In [1]: run delme.py
worker 29963, person name a, received int 0
worker 29962, person name b, received int 1
worker 29964, person name c, received int 2
worker 29962, person name d, received int 3
worker 29966, person name e, received int 4
worker 29967, person name f, received int 5
worker 29966, person name g, received int 6
worker 29967, person name h, received int 7
their number
0
1
2
3
4
5
6
7
False

---

我使用默认的多处理包，它没有这样的问题。

问题是self.num是在子进程中分配的。多处理不会将原始对象传递回调用方。它确实会传回方法的返回码。因此，您可以直接将num传递回，甚至传递回self，但这通常效率很低，不会替换父对象中的现有对象，只会创建一个新对象。

这里的问题非常类似：。答案基本上是@tdelaney在下面给出的扩展版本。您的编辑没有显示错误，这是一个功能。如果你在uuu main uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu。当您传递一个类实例时，默认情况下，dill会对实例和类定义进行pickle处理——因此，在从多处理返回时，您将从拾取的类生成一个新实例。pickle通过引用序列化，而dill默认情况下不会。使用dill，您可以通过引用序列化，从而引用原始类。pathos.multiprocessing使用dill。使用dill，通过将类定义与实例一起序列化…在取消pickle之前，您可以更改类定义、删除类定义或以其他方式禁用类定义，而实例仍将正确取消pickle。您甚至可以将该类实例解压到一个全新的python会话，在该会话中您没有导入或定义该类。关键是，这是默认设置，如果使用byref标志，则可以恢复到pickle行为。也许这有点出乎意料，但这是一个功能。如果你认为这不应该是一个功能，或者如果你认为该功能应该以某种方式修改，请提交一张票到我是dill and pathos的作者。有计划启用这种类型的行为，而且应该很简单……但是您描述的内容是正确的，您建议返回值是目前的做法。@MikeMcKerns-谢谢您提供的信息。我担心我可能错过了一些关于悲情的东西，所以我很高兴听到我做对了。更新父项将是一个有趣的功能。。。如果你不知道这件事正在发生，那就有可能是灾难性的！我完全同意。它是一个关键字选项，默认为False，可能仅限于特殊情况。不过，还不确定结果会如何。@MikeMcKerns，可以阅读问题的编辑，似乎pathos包中有一个bug。

In [1]: run delme.py
worker 29963, person name a, received int 0
worker 29962, person name b, received int 1
worker 29964, person name c, received int 2
worker 29962, person name d, received int 3
worker 29966, person name e, received int 4
worker 29967, person name f, received int 5
worker 29966, person name g, received int 6
worker 29967, person name h, received int 7
their number
0
1
2
3
4
5
6
7
False