用jq解析json树中的多个键/值

使用jq，我想从以下json中挑选键/值对：

{
  "project": "Project X",
  "description": "This is a description of Project X",
  "nodes": [
    {
      "name": "server001",
      "detail001": "foo",
      "detail002": "bar",
      "networks": [
        {
          "net_tier": "network_tier_001",
          "ip_address": "10.1.1.10",
          "gateway": "10.1.1.1",
          "subnet_mask": "255.255.255.0",
          "mac_address": "00:11:22:aa:bb:cc"
        }
      ],
      "hardware": {
        "vcpu": 1,
        "mem": 1024,
        "disks": [
          {
            "disk001": 40,
            "detail001": "foo"
          },
          {
            "disk002": 20,
            "detail001": "bar"
          }
        ]
      },
      "os": "debian8",
      "geo": {
        "region": "001",
        "country": "Sweden",
        "datacentre": "Malmo"
      },
      "detail003": "baz"
    }
  ],
  "detail001": "foo"
}

为了举个例子，我想解析以下键及其值：“Project”、“name”、“net_tier”、“vcpu”、“mem”、“disk001”、“disk002”

我能够解析单个元素而没有太多问题，但是由于完整解析的层次性，我没有太多机会解析不同的分支（即网络和硬件>磁盘）

谢谢你的帮助

编辑：

---

为清楚起见，我要使用的输出是逗号分隔的CSV。就解析所有组合而言，现在覆盖示例中的示例数据即可。我希望能够详细介绍任何建议。

这里有一种方法可以让您实现所需的输出

program.jq：

["project","name","net_tier","vcpu","mem","disk001","disk002"],
  [.project]
+ (.nodes[] | .networks[] as $n |
    [
      .name,
      $n.net_tier,
      (.hardware |
        .vcpu,
        .mem,
        (.disks | add["disk001","disk002"])
      )
    ]
  )
| @csv

---

基本上，您需要将所需的字段投影到数组中，以便将这些数组转换为csv行。您的输入使一个给定节点看起来可能有多个网络。因此，如果您想输出所有组合，则必须将其平展。

这里有一种方法可以实现所需的输出

program.jq：

["project","name","net_tier","vcpu","mem","disk001","disk002"],
  [.project]
+ (.nodes[] | .networks[] as $n |
    [
      .name,
      $n.net_tier,
      (.hardware |
        .vcpu,
        .mem,
        (.disks | add["disk001","disk002"])
      )
    ]
  )
| @csv

---

基本上，您需要将所需的字段投影到数组中，以便将这些数组转换为csv行。您的输入使一个给定节点看起来可能有多个网络。因此，如果要输出所有组合，必须将其平铺。

这里有另一种方法，它足够简短，足以说明问题：

def s(f): first(.. | f? // empty) // null;

[s(.project), s(.name), s(.net_tier), s(.vcpu), s(.mem), s(.disk001), s(.disk002)]
| @csv

调用：

$ jq -r -f value-pairs.jq input.json

结果:

"Project X","server001","network_tier_001",1,1024,40,20

带标题 使用与上述相同的

s/1

：

. as $d
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"]
| (., map( . as $v | $d | s(.[$v])))
| @csv

.project as $p
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"] as $h
| ($h,
   (.nodes[] as $d
   | $h
   | map( . as $v | $d | s(.[$v]) )
   | .[0] = $p)
   ) | @csv

具有多个节点 再次使用

s/1

如上所述：

. as $d
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"]
| (., map( . as $v | $d | s(.[$v])))
| @csv

.project as $p
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"] as $h
| ($h,
   (.nodes[] as $d
   | $h
   | map( . as $v | $d | s(.[$v]) )
   | .[0] = $p)
   ) | @csv

输出说明性多节点数据：

"project","name","net_tier","vcpu","mem","disk001","disk002"
"Project X","server001","network_tier_001",1,1024,40,20
"Project X","server002","network_tier_002",1,1024,,40

---

下面是另一种方法，它足够简短，足以说明问题：

def s(f): first(.. | f? // empty) // null;

[s(.project), s(.name), s(.net_tier), s(.vcpu), s(.mem), s(.disk001), s(.disk002)]
| @csv

调用：

$ jq -r -f value-pairs.jq input.json

结果:

"Project X","server001","network_tier_001",1,1024,40,20

带标题 使用与上述相同的

s/1

：

. as $d
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"]
| (., map( . as $v | $d | s(.[$v])))
| @csv

.project as $p
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"] as $h
| ($h,
   (.nodes[] as $d
   | $h
   | map( . as $v | $d | s(.[$v]) )
   | .[0] = $p)
   ) | @csv

具有多个节点 再次使用

s/1

如上所述：

. as $d
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"]
| (., map( . as $v | $d | s(.[$v])))
| @csv

.project as $p
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"] as $h
| ($h,
   (.nodes[] as $d
   | $h
   | map( . as $v | $d | s(.[$v]) )
   | .[0] = $p)
   ) | @csv

输出说明性多节点数据：

"project","name","net_tier","vcpu","mem","disk001","disk002"
"Project X","server001","network_tier_001",1,1024,40,20
"Project X","server002","network_tier_002",1,1024,,40

---

这里有一个不同的过滤器，它计算网络层和磁盘名称的唯一集合，然后生成一个结果，其中包含适合数据的列

  {
    tiers: [ .nodes[].networks[].net_tier ] | unique
  , disks: [ .nodes[].hardware.disks[] | keys[] | select(startswith("disk")) ] | unique
  } as $n

| def column_names($n): [ "project", "name" ] + $n.tiers + ["vcpu", "mem"] + $n.disks ;
  def tiers($n):        [ $n.tiers[] as $t | .networks[] | if .net_tier==$t then $t else null end ] ;
  def disks($n):        [ $n.disks[] as $d | map(select(.[$d]!=null)|.[$d])[0] ] ;
  def rows($n):
      .project as $project
    | .nodes[]
    | .name as $name
    | tiers($n) as $tier_values
    | .hardware
    | .vcpu as $vcpu
    | .mem as $mem
    | .disks
    | disks($n) as $disk_values
    | [$project, $name] + $tier_values + [$vcpu, $mem] + $disk_values
  ;
  column_names($n), rows($n)

| @csv

如果我们将另一个节点添加到样本数据中，这种方法的优点就会变得明显：

{
  "name": "server002",
  "networks": [
    {
      "net_tier": "network_tier_002"
    }
  ],
  "hardware": {
    "vcpu": 1,
    "mem": 1024,
    "disks": [
      {
        "disk002": 40,
        "detail001": "foo"
      }
    ]
  }
}

示例运行（假设在

filter.jq

中使用filter，在

data.json

中使用修改后的数据）

---

这里有一个不同的过滤器，它计算网络层和磁盘名称的唯一集合，然后生成一个结果，其中包含适合数据的列

  {
    tiers: [ .nodes[].networks[].net_tier ] | unique
  , disks: [ .nodes[].hardware.disks[] | keys[] | select(startswith("disk")) ] | unique
  } as $n

| def column_names($n): [ "project", "name" ] + $n.tiers + ["vcpu", "mem"] + $n.disks ;
  def tiers($n):        [ $n.tiers[] as $t | .networks[] | if .net_tier==$t then $t else null end ] ;
  def disks($n):        [ $n.disks[] as $d | map(select(.[$d]!=null)|.[$d])[0] ] ;
  def rows($n):
      .project as $project
    | .nodes[]
    | .name as $name
    | tiers($n) as $tier_values
    | .hardware
    | .vcpu as $vcpu
    | .mem as $mem
    | .disks
    | disks($n) as $disk_values
    | [$project, $name] + $tier_values + [$vcpu, $mem] + $disk_values
  ;
  column_names($n), rows($n)

| @csv

如果我们将另一个节点添加到样本数据中，这种方法的优点就会变得明显：

{
  "name": "server002",
  "networks": [
    {
      "net_tier": "network_tier_002"
    }
  ],
  "hardware": {
    "vcpu": 1,
    "mem": 1024,
    "disks": [
      {
        "disk002": 40,
        "detail001": "foo"
      }
    ]
  }
}

示例运行（假设在

filter.jq

中使用filter，在

data.json

中使用修改后的数据）

---

那么，您的预期产出是多少？您提到了字段，但您希望它是如何构造的？数据中还有一堆数组。如果有比示例中更多的内容，它将如何影响您的预期输出？您希望输出所有的组合吗？请在您的问题中更清楚地说明这一点。谢谢Jeff，为了清晰起见，我进行了编辑。那么，您的预期输出是什么呢？您提到了字段，但您希望它是如何构造的？数据中还有一堆数组。如果有比示例中更多的内容，它将如何影响您的预期输出？您希望输出所有的组合吗？请在你的问题中更清楚地说明这一点。谢谢杰夫，为了清楚起见，我已经编辑了。