解组json流(不以换行符分隔)
我想将JSON流转换为对象流。使用换行符分隔的JSON很容易做到这一点。从Go文档: 但是,我需要从JSON数组生成一个流,如下所示:解组json流(不以换行符分隔),json,go,stream,decode,Json,Go,Stream,Decode,我想将JSON流转换为对象流。使用换行符分隔的JSON很容易做到这一点。从Go文档: 但是,我需要从JSON数组生成一个流,如下所示: [{"Name": "Ed", "Text": "Knock knock."}, {"Name": "Sam", "Text": "Who's there?"},
[{"Name": "Ed", "Text": "Knock knock."},
{"Name": "Sam", "Text": "Who's there?"},
{"Name": "Ed", "Text": "Go fmt."},
{"Name": "Sam", "Text": "Go fmt who?"},
{"Name": "Ed", "Text": "Go fmt yourself!"}]
做这件事的最佳方式是什么
我考虑过这种方法:
我希望用嵌套对象和任意结构解析JSON。您可以使用流式解析器。例如:
这可能是您所能得到的最有效的,但它确实需要大量的手动处理。假设json流如下:
{"Name": "Ed", "Text": "Knock knock."}{"Name": "Sam", "Text": "Who's there?"}{"Name": "Ed", "Text": "Go fmt."}
我有一个想法,伪代码如下:
1: skip prefix whitespace
2: if first char not {, throw error
3: load some chars, and find the first "}"
4: if found, try json.Unmarshal()
5: if unmarshal fail, load more chars, and find second "}"
6: redo STEP 4
func process(src []byte) error {
s := json.NewStream(src)
for {
obj, err := s.Read()
switch err {
case nil:
case io.EOF:
return nil
default:
return err
}
// now you can try to decode the obj to a struct/map/...
// it is also support mix stream, ex.:
a = new(TypeOne)
b = new(TypeTwo)
if err := j.Unmarshal(obj, a); err == nil && a.Error != "" {
// it is a TypeOne object
} else if err := j.Unmarshal(obj, b); err == nil && a.ID != "" {
// it is a TypeTwo object
} else {
// unkown type
}
}
return nil
}
下面是一个已经在我的项目中运行的实现:
package json
import (
"bytes"
j "encoding/json"
"errors"
"io"
"strings"
)
// Stream represent a json stream
type Stream struct {
stream *bytes.Buffer
object *bytes.Buffer
scrap *bytes.Buffer
}
// NewStream return a Stream that based on src
func NewStream(src []byte) *Stream {
return &Stream{
stream: bytes.NewBuffer(src),
object: new(bytes.Buffer),
scrap: new(bytes.Buffer),
}
}
// Read read a json object
func (s *Stream) Read() ([]byte, error) {
var obj []byte
for {
// read a rune from stream
r, _, err := s.stream.ReadRune()
switch err {
case nil:
case io.EOF:
if strings.TrimSpace(s.object.String()) != "" {
return nil, errors.New("Invalid JSON")
}
fallthrough
default:
return nil, err
}
// write the rune to object buffer
if _, err := s.object.WriteRune(r); err != nil {
return nil, err
}
if r == '}' {
obj = s.object.Bytes()
// check whether json string valid
err := j.Compact(s.scrap, obj)
s.scrap.Reset()
if err != nil {
continue
}
s.object.Reset()
break
}
}
return obj, nil
}
用法如下:
1: skip prefix whitespace
2: if first char not {, throw error
3: load some chars, and find the first "}"
4: if found, try json.Unmarshal()
5: if unmarshal fail, load more chars, and find second "}"
6: redo STEP 4
func process(src []byte) error {
s := json.NewStream(src)
for {
obj, err := s.Read()
switch err {
case nil:
case io.EOF:
return nil
default:
return err
}
// now you can try to decode the obj to a struct/map/...
// it is also support mix stream, ex.:
a = new(TypeOne)
b = new(TypeTwo)
if err := j.Unmarshal(obj, a); err == nil && a.Error != "" {
// it is a TypeOne object
} else if err := j.Unmarshal(obj, b); err == nil && a.ID != "" {
// it is a TypeTwo object
} else {
// unkown type
}
}
return nil
}
是的,如果你知道格式是这样的,“预解析”一次提取和解码一个对象对我来说就像任何方法一样好。这是一些可靠的示例文本。:)示例文本来自Go文档-实际上直到现在才阅读!这很好,但我正在寻找能够处理具有任意结构的嵌套对象的东西。如果我能弄明白,我会更新这个问题并添加到你的代码中。第3步很危险:
加载一些字符,然后找到第一个“}”
如果“}”在引号内,我们不想停止扫描。示例:{“key”:“带}}}方括号的值”}
。您提供的伪代码是为一个幼稚的解析器提供的,但需要的是一个成熟的解析器。是的,步骤3是危险的,但步骤4/5/6将保证正确的工作。