如何在Python中将JSON字符串转换为Avro?
有没有一种方法可以将JSON字符串转换为Avro,而不需要Python中的模式定义?或者这是只有Java才能处理的事情?: 这将有助于:如何在Python中将JSON字符串转换为Avro?,python,avro,Python,Avro,有没有一种方法可以将JSON字符串转换为Avro,而不需要Python中的模式定义?或者这是只有Java才能处理的事情?: 这将有助于: b = BytesIO(b'some message') reader = DataFileReader(b, DatumReader()) 有关更多信息,请看一看。我最近遇到了同样的问题,我最终开发了一个python包,它可以接受任何python数据结构,包括解析的JSON,并将其存储在Avro中,而不需要专用的模式 我用python 3测试了它 您可以将
b = BytesIO(b'some message')
reader = DataFileReader(b, DatumReader())
有关更多信息,请看一看。我最近遇到了同样的问题,我最终开发了一个python包,它可以接受任何python数据结构,包括解析的JSON,并将其存储在Avro中,而不需要专用的模式 我用python 3测试了它 您可以将其安装为
pip3 install rec avro
,也可以在
用法示例:
from fastavro import writer, reader, schema
from rec_avro import to_rec_avro_destructive, from_rec_avro_destructive, rec_avro_schema
def json_objects():
return [{'a': 'a'}, {'b':'b'}]
# For efficiency, to_rec_avro_destructive() destroys rec, and reuses it's
# data structures to construct avro_objects
avro_objects = (to_rec_avro_destructive(rec) for rec in json_objects())
# store records in avro
with open('json_in_avro.avro', 'wb') as f_out:
writer(f_out, schema.parse_schema(rec_avro_schema()), avro_objects)
#load records from avro
with open('json_in_avro.avro', 'rb') as f_in:
# For efficiency, from_rec_avro_destructive(rec) destroys rec, and
# reuses it's data structures to construct it's output
loaded_json = [from_rec_avro_destructive(rec) for rec in reader(f_in)]
assert loaded_json == json_objects()
要将JSON字符串转换为JSON对象,请使用
JSON.loads('{“a”:“b”}')
这将解析avro模式,而不是序列化的avro文档。
from fastavro import writer, reader, schema
from rec_avro import to_rec_avro_destructive, from_rec_avro_destructive, rec_avro_schema
def json_objects():
return [{'a': 'a'}, {'b':'b'}]
# For efficiency, to_rec_avro_destructive() destroys rec, and reuses it's
# data structures to construct avro_objects
avro_objects = (to_rec_avro_destructive(rec) for rec in json_objects())
# store records in avro
with open('json_in_avro.avro', 'wb') as f_out:
writer(f_out, schema.parse_schema(rec_avro_schema()), avro_objects)
#load records from avro
with open('json_in_avro.avro', 'rb') as f_in:
# For efficiency, from_rec_avro_destructive(rec) destroys rec, and
# reuses it's data structures to construct it's output
loaded_json = [from_rec_avro_destructive(rec) for rec in reader(f_in)]
assert loaded_json == json_objects()