elasticsearch 在logstash管道中使用elasticsearch过滤器

我正在日志存储管道中使用elasticsearch过滤器。我使用以下方法正确找到结果：

filter{
  if [class] == "DPAPIINTERNAL" {
    elasticsearch {
      hosts => "10.1.10.16"
      index => "dp_audit-2017.02.16"
      query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
    }
  }
}

如您所见，我使用的“查询模板”是：

{
    "query": {
      "query_string": {
       "query": "class:DPAPI AND request.aw:%{[aw]}"
      }
    },
   "_source": ["end_point", "vittorio"]
 }

这告诉elastichsearch查找与DPAPIInteral日志匹配的特定类的日志

太好了！但是现在我找到了结果，我想从中添加一些字段并将它们附加到我的dpapinternal日志中，例如，我想获取“end_point”并将其添加到日志中的新键“vittorio”中

这并没有发生，我不明白为什么

下面是我正在使用查询查看的日志：

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "dp_audit-2017.02.16",
        "_type": "logs",
        "_id": "AVpHoPHPuEPlW12Qu",
        "_score": 1,
        "_source": {
          "svc": "dp-1.1",
          "request": {
            "method": "POST|PATCH|DELETE",
            "aw": "prova",
            "end_point": "/bank/6311",
            "app_instance": "7D1-D233-87E1-913"
          },
          "path": "/home/vittorio/Documents/dpapi1.json",
          "@timestamp": "2017-02-16T15:53:33.214Z",
          "@version": "1",
          "host": "Vito",
          "event": "bank.add",
          "class": "DPAPI",
          "ts": "2017-01-16T19:20:30.125+01:00"
        }
      }
    ]
  }
}

您需要在

elasticsearch

过滤器中指定

elasticsearch {
  hosts => "10.1.10.16"
  index => "dp_audit-2017.02.16"
  query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
  fields => { "[request][end_point]" => "vittorio" }
}

"_source": ["request.end_point"]

请注意，由于

结束点

是一个嵌套字段，因此需要在查询模板中修改

源

，如下所示：

elasticsearch {
  hosts => "10.1.10.16"
  index => "dp_audit-2017.02.16"
  query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
  fields => { "[request][end_point]" => "vittorio" }
}

"_source": ["request.end_point"]

---

问题很简单，您不必使用query_模板指定“new”字段

"_source": ["request"] # here you specify the field you want from the query result.

然后

filter{
  if [class] == "DPAPIINTERNAL" {
    elasticsearch {
      hosts => "10.1.10.16"
      index => "dp_audit-2017.02.16"
      query_template => "/home/vittorio/Documents/elastic-queries/matching-requestaw.json"
      fields => {"request" => "new_key"} # here you add the fields and will tell elastich filter to put request inside new_key
    }
  }
}

---

这对我有用

我已经在查询模板文件中指定了字段：“\u source”：[“end\u point”，“vittorio”]在模板中，您指定了要在搜索中检索的字段，但在

字段

参数中，您实际将这些字段分配给了事件。我刚刚指定了，我的新字段中有空值：（哦，我知道为什么，

end\u point

实际上是一个嵌套字段。我已经更新了我的答案。您确定所检索的文档中

“request.end\u point”

不为空吗？