Python 3.x Python3上cython编译对象和多处理的问题

我有一个使用cython编译的对象库。当我尝试在使用Python3和多进程处理（池）的简单脚本上使用这些对象时，python抛出了一个pickle错误。奇怪的是，如果我使用Python2，一切都会按预期工作（用Python2中的cython编译代码，用Python2运行代码）。此外，当我尝试使用一个简单的脚本在Python3上实际pickle cython对象（无需多处理）时，没有发现任何问题

以下是我的一个小例子：

示例库pyx:

'''
example_lib.pyx
'''

#import numpy as np

include "cython_object.pyx"

cdef class CythonObject:
    def __init__(self, str name, int a, int b):

        self._name = name
        self._a = a
        self._b = b

    property Name:
        '''
        Returns the name of the object.
        '''   
        def __get__(self):
            return self._name

    property A:
        '''
        Returns a.
        '''   
        def __get__(self):
            return self._a

    property B:
        '''
        Returns b.
        '''   
        def __get__(self):
            return self._b

    cpdef getSum(self):
        return self._a + self._b

    cpdef dict pack(self):
        cdef dict pack = { 'name': self._name
                         , 'a'   : self._a
                         , 'b'   : self._b
                         }
        return pack

    cpdef unPack(self, dict pack):
        self._name = str(pack['name'])
        self._a = int(pack['a'])
        self._b = int(pack['b'])

try:
    from setuptools import setup
    from setuptools import Extension
except ImportError:
    from distutils.core import setup
    from distutils.extension import Extension
from Cython.Build import cythonize
import numpy

setup(
    name = 'cythonobjects'
   ,ext_modules = cythonize("example_lib.pyx")
   ,include_dirs=[numpy.get_include()]
)     

cython\u object.pyx:

'''
example_lib.pyx
'''

#import numpy as np

include "cython_object.pyx"

cdef class CythonObject:
    def __init__(self, str name, int a, int b):

        self._name = name
        self._a = a
        self._b = b

    property Name:
        '''
        Returns the name of the object.
        '''   
        def __get__(self):
            return self._name

    property A:
        '''
        Returns a.
        '''   
        def __get__(self):
            return self._a

    property B:
        '''
        Returns b.
        '''   
        def __get__(self):
            return self._b

    cpdef getSum(self):
        return self._a + self._b

    cpdef dict pack(self):
        cdef dict pack = { 'name': self._name
                         , 'a'   : self._a
                         , 'b'   : self._b
                         }
        return pack

    cpdef unPack(self, dict pack):
        self._name = str(pack['name'])
        self._a = int(pack['a'])
        self._b = int(pack['b'])

try:
    from setuptools import setup
    from setuptools import Extension
except ImportError:
    from distutils.core import setup
    from distutils.extension import Extension
from Cython.Build import cythonize
import numpy

setup(
    name = 'cythonobjects'
   ,ext_modules = cythonize("example_lib.pyx")
   ,include_dirs=[numpy.get_include()]
)     

示例库设置：

'''
example_lib.pyx
'''

#import numpy as np

include "cython_object.pyx"

cdef class CythonObject:
    def __init__(self, str name, int a, int b):

        self._name = name
        self._a = a
        self._b = b

    property Name:
        '''
        Returns the name of the object.
        '''   
        def __get__(self):
            return self._name

    property A:
        '''
        Returns a.
        '''   
        def __get__(self):
            return self._a

    property B:
        '''
        Returns b.
        '''   
        def __get__(self):
            return self._b

    cpdef getSum(self):
        return self._a + self._b

    cpdef dict pack(self):
        cdef dict pack = { 'name': self._name
                         , 'a'   : self._a
                         , 'b'   : self._b
                         }
        return pack

    cpdef unPack(self, dict pack):
        self._name = str(pack['name'])
        self._a = int(pack['a'])
        self._b = int(pack['b'])

try:
    from setuptools import setup
    from setuptools import Extension
except ImportError:
    from distutils.core import setup
    from distutils.extension import Extension
from Cython.Build import cythonize
import numpy

setup(
    name = 'cythonobjects'
   ,ext_modules = cythonize("example_lib.pyx")
   ,include_dirs=[numpy.get_include()]
)     

test.py：

import multiprocessing
from example_lib import *

def runReplica(inputs):
    (i, objs) = inputs
    for obj,pack in objs:
        obj.unPack(pack)
        print('Rep:{}. Name: {}. {} + {} = {}'.format(i, obj.Name, obj.A, obj.B, obj.getSum()))

if __name__ == '__main__':

    obj1 = CythonObject('obj1', 2, 3)
    obj2 = CythonObject('obj2', 4, 2)

    objs = [obj1, obj2]

    num_rep = 3
    num_pool = 2

    packs = [(obj,obj.pack()) for obj in objs]
    if num_pool==1 or num_rep==1:
        for i in range(num_rep):
            runReplica((i,packs))
    else:
        pool = multiprocessing.Pool(num_pool)
        print ("Running parallelized replicas")
        pool.map(runReplica,[(i,packs) for i in range(num_rep)])

生成文件：

PYTHON=py -3
PYTHONVERSION=36
PYTHONHOME=C:\Python36
NUMPYINCLUDE=C:\Python36\lib\site-packages\numpy\core\include
CYTHON=cython.exe
CC=gcc
RM=rm
RMR= rm -r
MKDIR=mkdir
MKDIRR=mkdir -r
RMDIR=rmdir /s /q
CP=cp
UIC= pyside-uic
RCC= pyside-rcc
DEV=0
DEBUG=0

all:
    make example_pyd

example_pyd:
    $(PYTHON) example_lib_setup.py build_ext --inplace
    -$(RM) example_lib.c

test:
    $(PYTHON) test.py

clean:
ifeq ($(PYTHONVERSION),27)
    $(RM) example_lib.pyd
endif
ifeq ($(PYTHONVERSION),36)
    $(RM) example_lib.cp36-win_amd64.pyd
endif
    -$(RMR) build

这些都是在Windows10平台上使用Python3.6.7、Cython0.25.2、Python2.7.14和相同版本的cython（用于py2）进行测试的

为了使用python 2进行编译，请更改Makefile上的相关变量值

---

提前谢谢

你得到了什么错误？@MisterMiyagi:“TypeError:无法pickle example_lib.CythonObject对象”，问题是，如果我在多处理之外对对象进行pickle，则不会发生此类错误。在Windows 10上使用编译的cython库进行多处理时，我也会遇到同样的错误。Ubuntu上的Python2.7从来没有出现过这个问题。如果能澄清如何在这里继续，那就太好了。@phasselmann您应该重写对象“”\uuuuuGetState\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu。我认为这是酸洗错误。我正在处理的函数是Cython，它基本上是一个在类中调用函数的循环。。。你能告诉我怎么做吗？非常感谢。您得到了什么错误？@MisterMiyagi:“TypeError:无法pickle example_lib.CythonObject对象”，问题是，如果我在多处理之外对对象进行pickle，则不会发生此类错误。在Windows 10上使用带多处理的已编译cython库时，我也会遇到同样的错误。Ubuntu上的Python2.7从来没有出现过这个问题。如果能澄清如何在这里继续，那就太好了。@phasselmann您应该重写对象“”\uuuuuGetState\uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu。我认为这是酸洗错误。我正在处理的函数是Cython，它基本上是一个在类中调用函数的循环。。。你能告诉我怎么做吗？谢谢。