Python 使用ruamel.yaml安全倾倒和装载defaultdict
我正在尝试(反)序列化Python中具有Python 使用ruamel.yaml安全倾倒和装载defaultdict,python,yaml,defaultdict,ruamel.yaml,Python,Yaml,Defaultdict,Ruamel.yaml,我正在尝试(反)序列化Python中具有collections.defaultdict属性的类(在我的例子中是3.6+) 这是我想开始工作的一个最小的例子: from collections import defaultdict import ruamel.yaml from pathlib import Path class Foo: def __init__(self): self.x = defaultdict() YAML = ruamel.yaml.YAM
collections.defaultdict
属性的类(在我的例子中是3.6+)
这是我想开始工作的一个最小的例子:
from collections import defaultdict
import ruamel.yaml
from pathlib import Path
class Foo:
def __init__(self):
self.x = defaultdict()
YAML = ruamel.yaml.YAML(typ="safe")
YAML.register_class(Foo)
YAML.register_class(defaultdict)
fp = Path("./test.yaml")
YAML.dump(Foo(), fp)
YAML.load(fp)
但这在以下方面失败:
AttributeError:'collections.defaultdict'对象没有属性'\uu dict'
有没有什么想法不需要为每个“Foo-like”类编写自定义代码?我希望能为defaultdict
对象添加一个不同的representer,但到目前为止我的尝试都是徒劳的
完全回溯:
Traceback (most recent call last):
File "./tests/test_yaml.py", line 18, in <module>
YAML.dump(Foo(), fp)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\main.py", line 439, in dump
return self.dump_all([data], stream, _kw, transform=transform)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\main.py", line 453, in dump_all
self._context_manager.dump(data)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\main.py", line 801, in dump
self._yaml.representer.represent(data)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 81, in represent
node = self.represent_data(data)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 108, in represent_data
node = self.yaml_representers[data_types[0]](self, data)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\main.py", line 638, in t_y
tag, data, cls, flow_style=representer.default_flow_style
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 384, in represent_yaml_object
return self.represent_mapping(tag, state, flow_style=flow_style)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 218, in represent_mapping
node_value = self.represent_data(item_value)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 108, in represent_data
node = self.yaml_representers[data_types[0]](self, data)
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\main.py", line 638, in t_y
tag, data, cls, flow_style=representer.default_flow_style
File "C:\miniconda-windows\envs\ratio\lib\site-packages\ruamel\yaml\representer.py", line 383, in represent_yaml_object
state = data.__dict__.copy()
AttributeError: 'collections.defaultdict' object has no attribute '__dict__'
回溯(最近一次呼叫最后一次):
文件“/tests/test_yaml.py”,第18行,在
YAML.dump(Foo(),fp)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\main.py”,第439行,在转储文件中
返回self.dump_all([data],流,_kw,transform=transform)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\main.py”,第453行,位于dump\u all中
self.\u context\u manager.dump(数据)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\main.py”,第801行,在转储文件中
self.\u yaml.representer.represente(数据)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第81行,以
节点=自身。表示_数据(数据)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第108行,代表\u数据
node=self.yaml\u代表[data\u types[0]](self,data)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\main.py”,第638行,t\U y
标记、数据、cls、流\u样式=representer.default\u流\u样式
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第384行,在represente\u yaml\u对象中
返回self.representation\u映射(标记、状态、流样式=流样式)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第218行,在表示映射中
节点\u值=自身。表示\u数据(项目\u值)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第108行,代表\u数据
node=self.yaml\u代表[data\u types[0]](self,data)
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\main.py”,第638行,t\U y
标记、数据、cls、流\u样式=representer.default\u流\u样式
文件“C:\miniconda windows\envs\ratio\lib\site packages\ruamel\yaml\representer.py”,第383行,在represente\u yaml\u对象中
状态=数据。_udict__uu.copy()
AttributeError:'collections.defaultdict'对象没有属性'\uuu dict''
这是因为defaultdict
是内置类dict
的一个子类,该类没有用于YAML编码器生成类属性名称的\uuu dict\uuu
属性。在这种情况下,defaultdict
应被视为dict
,但问题是ruamel.yaml.representer.BaseRepresenter
类的方法只查看对象本身的类,以确定对象是否有representer:
data_types = type(data).__mro__
# ...skipped
if data_types[0] in self.yaml_representers:
node = self.yaml_representers[data_types[0]](self, data)
相反,应该做的是检查\uuu mro\uuuu
中的任何数据类型是否具有重新输入,并在发现以下数据类型时使用它:
if any(data_type in self.yaml_representers for data_type in data_types):
node = self.yaml_representers[next(data_type for data_type in data_types if data_type in self.yaml_representers)](self, data)
所以我们可以自己修补这个方法:
def represent_data(self, data):
# type: (Any) -> Any
if self.ignore_aliases(data):
self.alias_key = None
else:
self.alias_key = id(data)
if self.alias_key is not None:
if self.alias_key in self.represented_objects:
node = self.represented_objects[self.alias_key]
# if node is None:
# raise RepresenterError(
# "recursive objects are not allowed: %r" % data)
return node
# self.represented_objects[alias_key] = None
self.object_keeper.append(data)
data_types = type(data).__mro__
if representer.PY2:
# if type(data) is types.InstanceType:
if isinstance(data, representer.types.InstanceType):
data_types = representer.get_classobj_bases(data.__class__) + list(data_types)
if any(data_type in self.yaml_representers for data_type in data_types):
node = self.yaml_representers[next(data_type for data_type in data_types if data_type in self.yaml_representers)](self, data)
else:
for data_type in data_types:
if data_type in self.yaml_multi_representers:
node = self.yaml_multi_representers[data_type](self, data)
break
else:
if None in self.yaml_multi_representers:
node = self.yaml_multi_representers[None](self, data)
elif None in self.yaml_representers:
node = self.yaml_representers[None](self, data)
else:
node = representer.ScalarNode(None, representer.text_type(data))
# if alias_key is not None:
# self.represented_objects[alias_key] = node
return node
representer.BaseRepresenter.represent_data = represent_data
这样您的代码就可以在不注册defaultdict的情况下工作:
class Foo:
def __init__(self):
self.x = defaultdict()
YAML = ruamel.yaml.YAML(typ="safe")
YAML.register_class(Foo)
# YAML.register_class(defaultdict)
fp = Path("/temp/test.yaml")
YAML.dump(Foo(), fp)
YAML.load(fp)
from ruamel.yaml import representer
representer.SafeRepresenter.add_representer(defaultdict, representer.SafeRepresenter.represent_dict)
EDIT:一个更优雅的解决方案是简单地添加SafeRepresenter。将\u dict
方法表示为defaultdict
的representer:
class Foo:
def __init__(self):
self.x = defaultdict()
YAML = ruamel.yaml.YAML(typ="safe")
YAML.register_class(Foo)
# YAML.register_class(defaultdict)
fp = Path("/temp/test.yaml")
YAML.dump(Foo(), fp)
YAML.load(fp)
from ruamel.yaml import representer
representer.SafeRepresenter.add_representer(defaultdict, representer.SafeRepresenter.represent_dict)
现在有一个包
ruamel.yaml.pytypes
支持转储defaultdict
实例。请注意,如果您提供函数作为参数(用于默认工厂
),则需要指定typ='safe'
,否则无法表示工厂函数
在virtualenv中安装ruamel.yaml.pytypes
和ruamel.yaml
后,您可以执行以下操作:
yaml = ruamel.yaml.YAML(typ=['unsafe', 'pytypes'])
yaml.default_flow_style = False
buf = ruamel.yaml.compat.StringIO()
def factory():
import datetime
return datetime.datetime.now()
data = defaultdict(factory)
x = data[4]
data[2] = 42
yaml.dump(data, buf)
print(buf.getvalue(), end='')
d = yaml.load(buf.getvalue())
assert data == d
assert data.default_factory == d.default_factory
上面的内容将打印出来(您的日期时间将不同)
(断言不会引发异常)
有关获得类似结果的“手动”方法,请参见编辑历史记录。感谢您的明确解释。我们可以通过为
defaultdict
添加一个representer来实现这一点吗?我相信这会使解决方案稍微不那么“黑客”。如果它应该被视为一个常规的口述,我们应该能够重新使用其中的一些权利?确实如此。我已经用一个更优雅的解决方案更新了我的答案。请忽略猴子补丁。:-)以这种方式使用简单的add_representer的缺点是,在加载时,转储文件将不包含有关重新创建defaultdict的信息。您将得到一个正常的dict,因为该文件没有!defaultdict
标记。我更新了我的答案,以处理非无默认工厂
参数(特别是如何处理defaultdict(列表)
),如另一个答案中所示,ruamel内置了一个representer:from ruamel.yaml import representer representer.SafeRepresenter.add\u representer(defaultdict,representer.SafeRepresenter.Represente_dict)
ruamel是一个包名称空间,没有representer。ruamel.yaml.pytypes内置了defaultdict的representer。在这里重复的内容没有指出缺点,就是将defaultdict当作正常的dict转储,在过程中丢失信息。