使用jq将JSON转换为CSV
我正在尝试使用使用jq将JSON转换为CSV,json,linux,bash,csv,jq,Json,Linux,Bash,Csv,Jq,我正在尝试使用jq提取我的JSON文件中的sids、ll、state、name、smry值,并导出到csv JSON文件(out.JSON): { "data": [ { "meta": { "uid": 74529, "ll": [ -66.9333, 47.0667 ],
jq
提取我的JSON
文件中的sids、ll、state、name、smry
值,并导出到csv
JSON文件(out.JSON):
{
"data": [
{
"meta": {
"uid": 74529,
"ll": [
-66.9333,
47.0667
],
"sids": [
"CA008102500 6"
],
"state": "NB",
"elev": 1250,
"name": "LONG LAKE"
},
"smry": [
[
"42",
"1955-02-23"
]
]
},
{
"meta": {
"uid": 74534,
"ll": [
-67.2333,
45.9667
],
"sids": [
"CA008103425 6"
],
"state": "NB",
"elev": 150.9,
"name": "NACKAWIC"
},
"smry": [
[
"40",
"1969-02-23"
]
]
},
{
"meta": {
"uid": 74549,
"ll": [
-67.4667,
47.4667
],
"sids": [
"CA008104933 6"
],
"state": "NB",
"elev": 794,
"name": "ST QUENTIN"
},
"smry": [
[
"M",
"M"
]
]
},
{
"meta": {
"uid": 74550,
"ll": [
-67.2667,
45.1833
],
"sids": [
"CA008104936 6"
],
"state": "NB",
"elev": 36.1,
"name": "ST STEPHEN"
},
"smry": [
[
"48",
"1900-02-23"
]
]
},
{
"meta": {
"uid": 74554,
"ll": [
-67.25,
47.2667
],
"sids": [
"CA008105000 6"
],
"state": "NB",
"elev": 915.4,
"name": "SISSON DAM"
},
"smry": [
[
"35",
"1955-02-23"
]
]
}
]
}
jq '.data | [ {sids, ll, state, name, smry} ]' out.json
sids, ll, state, name, smry
CA008102500, -66.9333, 47.0667, NB, LONG LAKE, 42,1955-02-23
CA008103425, -67.2333, 45.9667, NB, NACKAWIC, 35,1955-02-23
终端代码:
{
"data": [
{
"meta": {
"uid": 74529,
"ll": [
-66.9333,
47.0667
],
"sids": [
"CA008102500 6"
],
"state": "NB",
"elev": 1250,
"name": "LONG LAKE"
},
"smry": [
[
"42",
"1955-02-23"
]
]
},
{
"meta": {
"uid": 74534,
"ll": [
-67.2333,
45.9667
],
"sids": [
"CA008103425 6"
],
"state": "NB",
"elev": 150.9,
"name": "NACKAWIC"
},
"smry": [
[
"40",
"1969-02-23"
]
]
},
{
"meta": {
"uid": 74549,
"ll": [
-67.4667,
47.4667
],
"sids": [
"CA008104933 6"
],
"state": "NB",
"elev": 794,
"name": "ST QUENTIN"
},
"smry": [
[
"M",
"M"
]
]
},
{
"meta": {
"uid": 74550,
"ll": [
-67.2667,
45.1833
],
"sids": [
"CA008104936 6"
],
"state": "NB",
"elev": 36.1,
"name": "ST STEPHEN"
},
"smry": [
[
"48",
"1900-02-23"
]
]
},
{
"meta": {
"uid": 74554,
"ll": [
-67.25,
47.2667
],
"sids": [
"CA008105000 6"
],
"state": "NB",
"elev": 915.4,
"name": "SISSON DAM"
},
"smry": [
[
"35",
"1955-02-23"
]
]
}
]
}
jq '.data | [ {sids, ll, state, name, smry} ]' out.json
sids, ll, state, name, smry
CA008102500, -66.9333, 47.0667, NB, LONG LAKE, 42,1955-02-23
CA008103425, -67.2333, 45.9667, NB, NACKAWIC, 35,1955-02-23
我收到以下错误:
assertion "cb == jq_util_input_next_input_cb" failed: file "/usr/src/ports/jq/jq-1.5-3.x86_64/src/jq-1.5/util.c", line 371, function: jq_util_input_get_position
Aborted (core dumped)
预期输出示例:
{
"data": [
{
"meta": {
"uid": 74529,
"ll": [
-66.9333,
47.0667
],
"sids": [
"CA008102500 6"
],
"state": "NB",
"elev": 1250,
"name": "LONG LAKE"
},
"smry": [
[
"42",
"1955-02-23"
]
]
},
{
"meta": {
"uid": 74534,
"ll": [
-67.2333,
45.9667
],
"sids": [
"CA008103425 6"
],
"state": "NB",
"elev": 150.9,
"name": "NACKAWIC"
},
"smry": [
[
"40",
"1969-02-23"
]
]
},
{
"meta": {
"uid": 74549,
"ll": [
-67.4667,
47.4667
],
"sids": [
"CA008104933 6"
],
"state": "NB",
"elev": 794,
"name": "ST QUENTIN"
},
"smry": [
[
"M",
"M"
]
]
},
{
"meta": {
"uid": 74550,
"ll": [
-67.2667,
45.1833
],
"sids": [
"CA008104936 6"
],
"state": "NB",
"elev": 36.1,
"name": "ST STEPHEN"
},
"smry": [
[
"48",
"1900-02-23"
]
]
},
{
"meta": {
"uid": 74554,
"ll": [
-67.25,
47.2667
],
"sids": [
"CA008105000 6"
],
"state": "NB",
"elev": 915.4,
"name": "SISSON DAM"
},
"smry": [
[
"35",
"1955-02-23"
]
]
}
]
}
jq '.data | [ {sids, ll, state, name, smry} ]' out.json
sids, ll, state, name, smry
CA008102500, -66.9333, 47.0667, NB, LONG LAKE, 42,1955-02-23
CA008103425, -67.2333, 45.9667, NB, NACKAWIC, 35,1955-02-23
我做错了什么?这有点复杂,因为需要展平
SID
、ll
和smry
,然后才能展平整个记录。我建议创建一个jq
文件:
foo.jq:
然后打电话:
jq -rf foo.jq file.json
输出:
CA008102500,-66.9333,47.0667,NB,LONG LAKE,42,1955-02-23
CA008103425,-67.2333,45.9667,NB,NACKAWIC,40,1969-02-23
CA008104933,-67.4667,47.4667,NB,ST QUENTIN,M,M
CA008104936,-67.2667,45.1833,NB,ST STEPHEN,48,1900-02-23
CA008105000,-67.25,47.2667,NB,SISSON DAM,35,1955-02-23
JSON中有很多结构,您在尝试时忽略了这些结构。不过,您可能正在使用旧版本的
jq
;我得到一个简单的错误jq:error(在tmp.json:6):无法用字符串“sids”索引数组,而不是核心转储。Hi@chepner!我不太熟悉json
,但我已经认识到了文件的复杂树结构。@arnpry:jq——原始输出.data[]\124;.meta“\(.sids)\(.ll)\(.state)\(.name)”
,给出了值,但是我没有足够的知识来整平数组。如果任何程序转储了它的核心,而你能够复制它,那么这个程序就做错了,而不是你。您应该在jq
1.5中不发生内核转储时向上游报告错误;如果有一个bug,它已经被修复了。这看起来就像我正在尝试做的!但是得到以下错误:“jq:error:syntax error,意外的无效字符(unixshell引用问题?)在第1行:jq:1编译错误”我真的不相信这一点。你确定你复制的命令是正确的吗?哇!我使用记事本++创建了该文件,当我保存该文件时,EOL转换设置为Windows(添加额外的换行符),而不是Unix。与@csv
@CharlesDuffy True相比,使用join(“,”
处理角点情况较差。但是,join(“,”)
对于非拐角的情况是很好的,因为您可以轻松地通过管道连接到awk-F,
。但是是的,在一个健壮的脚本中,我会使用@csv