如何使用Python将JSON对象解析为更小的对象？

我有一个非常大的JSON对象，我需要将其拆分成更小的对象，并将这些较小的对象写入文件

样本数据

raw = '[{"id":"1","num":"2182","count":-17}{"id":"111","num":"3182","count":-202}{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'

所需输出（在本例中，将数据一分为二）

当前代码

import pandas as pd
import itertools
import json
from itertools import zip_longest


def grouper(iterable, n, fillvalue=None):
    args = [iter(iterable)] * n
    return zip_longest(fillvalue=fillvalue, *args)

    raw = '[{"id":"1","num":"2182","count":-17}{"id":"111","num":"3182","count":-202}{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'

#split the data into manageable chunks + write to files

for i, group in enumerate(grouper(raw, 4)):
    with open('outputbatch_{}.json'.format(i), 'w') as outputfile:
        json.dump(list(group), outputfile)

第一个文件“outputbatch_0.json”的当前输出

---

我觉得这比需要的要困难得多。

假设raw应该是一个有效的json字符串（我包括了缺少的逗号），下面是一个简单但有效的解决方案

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'
json_data = json.loads(raw)

def split_in_files(json_data, amount):
    step = len(json_data) // amount
    pos = 0
    for i in range(amount - 1):
        with open('output_file{}.json'.format(i+1), 'w') as file:
            json.dump(json_data[pos:pos+step], file)
            pos += step
    # last one
    with open('output_file{}.json'.format(amount), 'w') as file:
        json.dump(json_data[pos:], file)

split_in_files(json_data, 2)

---

假设raw应该是一个有效的json字符串（我包括了缺少的逗号），下面是一个简单但有效的解决方案

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'
json_data = json.loads(raw)

def split_in_files(json_data, amount):
    step = len(json_data) // amount
    pos = 0
    for i in range(amount - 1):
        with open('output_file{}.json'.format(i+1), 'w') as file:
            json.dump(json_data[pos:pos+step], file)
            pos += step
    # last one
    with open('output_file{}.json'.format(amount), 'w') as file:
        json.dump(json_data[pos:], file)

split_in_files(json_data, 2)

---

如果raw是有效的json。保存部分不详细

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'

raw_list = eval(raw)
raw__zipped = list(zip(raw_list[0::2], raw_list[1::2]))

for item in raw__zipped:
    with open('a.json', 'w') as f:
        json.dump(item, f)

---

如果raw是有效的json。保存部分不详细

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'

raw_list = eval(raw)
raw__zipped = list(zip(raw_list[0::2], raw_list[1::2]))

for item in raw__zipped:
    with open('a.json', 'w') as f:
        json.dump(item, f)

---

如果需要正好一半的数据，可以使用切片：

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'
json_data = json.loads(raw)

size_of_half = len(json_data)/2

print json_data[:size_of_half]
print json_data[size_of_half:]

---

在共享代码中，基本情况不需要处理，例如如果长度为奇数等，简而言之，您可以使用列表执行所有操作。

如果您需要正好一半的数据，可以使用切片：

import json

raw = '[{"id":"1","num":"2182","count":-17},{"id":"111","num":"3182","count":-202},{"id":"222","num":"4182","count":12},{"id":"33333","num":"5182","count":12}]'
json_data = json.loads(raw)

size_of_half = len(json_data)/2

print json_data[:size_of_half]
print json_data[size_of_half:]

---

在共享代码中，基本情况不会被处理，例如如果长度是奇数等等，简而言之，您可以对列表执行所有操作。

您的

raw

字符串不是有效的JSON（对象之间缺少逗号）。您的真实数据是这样的还是问题中的一个输入错误？您的

raw

字符串不是有效的JSON（对象之间缺少逗号）。你的真实数据是这样的，还是只是问题中的一个输入错误？