Python-dill:Can';泡菜装饰班
我有以下代码,用于装饰类:Python-dill:Can';泡菜装饰班,python,class,decorator,pickle,dill,Python,Class,Decorator,Pickle,Dill,我有以下代码,用于装饰类: import dill from collections import namedtuple from multiprocessing import Process def proxified(to_be_proxied): b = namedtuple('d', []) class Proxy(to_be_proxied, b): pass Proxy.__name__ = to_be_proxied.__name__
import dill
from collections import namedtuple
from multiprocessing import Process
def proxified(to_be_proxied):
b = namedtuple('d', [])
class Proxy(to_be_proxied, b):
pass
Proxy.__name__ = to_be_proxied.__name__
return Proxy
@proxified
class MyClass:
def f(self):
print('hello!')
pickled_cls = dill.dumps(MyClass)
def new_process(clazz):
dill.loads(clazz)().f()
p = Process(target=new_process, args=(pickled_cls,))
p.start()
p.join()
当我尝试pickle装饰类时,我得到以下错误:
Traceback (most recent call last):
File "/usr/lib/python3.5/pickle.py", line 907, in save_global
obj2, parent = _getattribute(module, name)
File "/usr/lib/python3.5/pickle.py", line 265, in _getattribute
.format(name, obj))
AttributeError: Can't get local attribute 'proxified.<locals>.Proxy' on <function proxified at 0x7fbf7de4b8c8>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/carbolymer/example.py", line 108, in <module>
pickled_cls = dill.dumps(MyClass)
File "/usr/lib/python3.5/site-packages/dill/dill.py", line 243, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
File "/usr/lib/python3.5/site-packages/dill/dill.py", line 236, in dump
pik.dump(obj)
File "/usr/lib/python3.5/pickle.py", line 408, in dump
self.save(obj)
File "/usr/lib/python3.5/pickle.py", line 475, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python3.5/site-packages/dill/dill.py", line 1189, in save_type
StockPickler.save_global(pickler, obj)
File "/usr/lib/python3.5/pickle.py", line 911, in save_global
(obj, module_name, name))
_pickle.PicklingError: Can't pickle <class '__main__.proxified.<locals>.Proxy'>: it's not found as __main__.proxified.<locals>.Proxy
回溯(最近一次呼叫最后一次):
文件“/usr/lib/python3.5/pickle.py”,第907行,在save_global中
obj2,父项=_getattribute(模块,名称)
文件“/usr/lib/python3.5/pickle.py”,第265行,在_getattribute中
.格式(名称,obj))
AttributeError:无法在上获取本地属性“proxified..Proxy”
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/home/carbolymer/example.py”,第108行,in
腌渍蔬菜=莳萝汤(MyClass)
文件“/usr/lib/python3.5/site packages/dill/dill.py”,第243行,转储
转储(对象、文件、协议、byref、fmode、递归)#、严格)
文件“/usr/lib/python3.5/site packages/dill/dill.py”,第236行,在转储中
排土场(obj)
文件“/usr/lib/python3.5/pickle.py”,第408行,在转储中
自我保存(obj)
文件“/usr/lib/python3.5/pickle.py”,第475行,保存
f(self,obj)#用显式self调用未绑定方法
文件“/usr/lib/python3.5/site packages/dill/dill.py”,第1189行,保存类型
StockPickler.save_global(pickler,obj)
文件“/usr/lib/python3.5/pickle.py”,第911行,保存在全局
(obj,模块名称,名称))
_pickle.PicklingError:无法pickle:找不到它作为_main__.proxified..Proxy
我如何用莳萝腌制装饰过的班级?我想把这个类作为一个参数传递给一个单独的进程——也许有更简单的方法吗?Gaël Varoquaux提供了一个关于“为什么酸洗修饰函数是痛苦的”的很好的解释
基本上,使用
functools.wrapps重写类可以避免这些问题:)在本例中,您有很多事情要做,其中许多很难序列化。首先,我建议使用multiprocess
而不是multiprocessing
,因此您可以利用dill
而不是pickle
。其次,尽可能避免使用namedtuple
。它们将序列化,但仅在某些情况下有效。第三,尽可能避免使用类装饰器。我不记得迪尔和他们相处得有多好,我觉得很糟糕。最后,您可以尝试不同的dill.settings
,了解dill
如何处理全局变量。首先,请使用多进程
,不要使用名称双工
。