Google bigquery 将json数组插入bigQuery时出错
读取数据时出错,错误消息:解析JSON失败:启动新数组时未找到对象。;BeginArray返回false 我已经创建了以下示例json数据。这是json的数组。每个json对象都在新行中 当我只加载一个json对象而不保留在数组中时,它就工作了 JSON正文-Google bigquery 将json数组插入bigQuery时出错,google-bigquery,Google Bigquery,读取数据时出错,错误消息:解析JSON失败:启动新数组时未找到对象。;BeginArray返回false 我已经创建了以下示例json数据。这是json的数组。每个json对象都在新行中 当我只加载一个json对象而不保留在数组中时,它就工作了 JSON正文- [ { "item_name": "dfkhjf", "gtin": "123456", "brand": "Om-Publication","category_name": "books", "country_code": "IN"
[
{ "item_name": "dfkhjf", "gtin": "123456", "brand": "Om-Publication","category_name": "books", "country_code": "IN", "marktet_place": "india", "price": 2239, "sellerId": 234, "create_time": "2017-07-19T16:00:46.000Z" },
{ "item_name": "toy-gun", "gtin": "1234234445", "brand": "Toy", "category_name": "toy", "country_code": "IN", "marktet_place": "flipMe", "price": 2239, "sellerId": 234, "create_time": "2017-08-19T16:00:46.000Z" },
{ "item_name": "Drone", "gtin": "12342356456", "brand": "Drone-XX", "category_name": "drone", "country_code": "IN", "marktet_place": "drone-maker", "price": 2239, "sellerId": 234, "create_time": "2017-09-19T16:00:46.000Z" }
]
正如的文档中所解释的,JSON数据必须在中,其中每一行都是有效的独立JSON值,因此应该使用(2)而不是(1): (一) (二)
更新: 下面是一个分步指南,以说明其工作原理: 使用我共享的内容创建一个JSON文件(
file.JSON
)(确保删除每行末尾的数组括号[]
,以及逗号,
:
$ cat file.json
{ "item_name": "dfkhjf", "gtin": "123456", "brand": "Om-Publication","category_name": "books", "country_code": "IN", "marktet_place": "india", "price": 2239, "sellerId": 234, "create_time": "2017-07-19T16:00:46.000Z" }
{ "item_name": "toy-gun", "gtin": "1234234445", "brand": "Toy", "category_name": "toy", "country_code": "IN", "marktet_place": "flipMe", "price": 2239, "sellerId": 234, "create_time": "2017-08-19T16:00:46.000Z" }
{ "item_name": "Drone", "gtin": "12342356456", "brand": "Drone-XX", "category_name": "drone", "country_code": "IN", "marktet_place": "drone-maker", "price": 2239, "sellerId": 234, "create_time": "2017-09-19T16:00:46.000Z" }
运行如下命令将文件加载到BQ:
$ bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON dataset.table file.json
Upload complete.
Waiting on bqjob_XXXXXXXXXXX ... (1s) Current status: DONE
现在查询该表以检查内容是否正确上载:
$ bq query --use_legacy_sql=false "SELECT * FROM dataset.table"
Waiting on bqjob_r3ef14ac0d0a6c856_000001681819e9fc_1 ... (0s) Current status: DONE
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+
| sellerId | price | create_time | marktet_place | brand | country_code | category_name | gtin | item_name |
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+
| 234 | 2239 | 2017-07-19 16:00:46 | india | Om-Publication | IN | books | 123456 | dfkhjf |
| 234 | 2239 | 2017-08-19 16:00:46 | flipMe | Toy | IN | toy | 1234234445 | toy-gun |
| 234 | 2239 | 2017-09-19 16:00:46 | drone-maker | Drone-XX | IN | drone | 12342356456 | Drone |
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+
您能解释一下如何执行insert命令并提供一个示例吗?我已将此数据上传到一个文件中,比如GCS bucket中的-mydata.json。然后从google控制台webUI创建表保持模式auto detect true。并从GCS上传此文件。如果您想在BigQuery中获得3行,可以删除[]从你的JSON,它应该会工作,我试过了。它的力量work@jyoti我已经更新了我的答案,显示了使加载工作正常的一步一步的过程。如果您仍然遇到这方面的问题,请尽可能具体地说明您看到的问题/错误。如果json对象具有完全相同的cont,OP的加载工作是否正常工作但是在一行中?我知道一旦它在bq中,你就必须取消它。
$ bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON dataset.table file.json
Upload complete.
Waiting on bqjob_XXXXXXXXXXX ... (1s) Current status: DONE
$ bq query --use_legacy_sql=false "SELECT * FROM dataset.table"
Waiting on bqjob_r3ef14ac0d0a6c856_000001681819e9fc_1 ... (0s) Current status: DONE
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+
| sellerId | price | create_time | marktet_place | brand | country_code | category_name | gtin | item_name |
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+
| 234 | 2239 | 2017-07-19 16:00:46 | india | Om-Publication | IN | books | 123456 | dfkhjf |
| 234 | 2239 | 2017-08-19 16:00:46 | flipMe | Toy | IN | toy | 1234234445 | toy-gun |
| 234 | 2239 | 2017-09-19 16:00:46 | drone-maker | Drone-XX | IN | drone | 12342356456 | Drone |
+----------+-------+---------------------+---------------+----------------+--------------+---------------+-------------+-----------+