elasticsearch 在使用grok logstash进行分析时为空字段插入伪值

我试图解析日志，并使用logstash将其放入弹性搜索中

我的日志文件的格式如下

[18-Aug-2016 02:28:46,537][ERROR][thread1][package.name] there is error in line 52

\[%{GREEDYDATA:date} %{GREEDYDATA:time}\]\[%{LOGLEVEL:log_type}\]\[%{GREEDYDATA:thread_name}\]\[%{GREEDYDATA:package}\](%{GREEDYDATA:log_msg})?

当我运行这个grok过滤器时，我得到了正确的输出。但是，在有些情况下，我获取的输入没有最后一个字段（log_msg）。大概是这样的：

[18-Aug-2016 02:28:46,537][ERROR][thread1][package.name]

在本例中，grok忽略了最后一个字段log_msg，而这并没有插入到弹性搜索中

但是，如果消息中不存在log_msg字段，我们可以设置一个空字符串或表示“no data”的字符串

实际输出：

{
        "message" => "[18-Aug-2016 02:28:46,537][ERROR][thread1][package.name]",
       "@version" => "1",
     "@timestamp" => "2016-08-17T12:31:58.209Z",
           "path" => "/home/admin-nfv/test1_log.log",
           "host" => "nendc1-bg-d104",
           "date" => "18-Aug-2016",
           "time" => "02:28:46,537",
       "log_type" => "ERROR",
    "thread_name" => "thread1",
        "package" => "package.name"
}

预期产出：

{
        "message" => "[18-Aug-2016 02:28:46,537][ERROR][thread1][package.name]",
       "@version" => "1",
     "@timestamp" => "2016-08-17T12:31:58.209Z",
           "path" => "/home/admin-nfv/test1_log.log",
           "host" => "nendc1-bg-d104",
           "date" => "18-Aug-2016",
           "time" => "02:28:46,537",
       "log_type" => "ERROR",
    "thread_name" => "thread1",
        "package" => "package.name",
        "log_msg" => "no data"
}

---

您可以添加一个

mutate

过滤器，该过滤器将在不存在空字段时添加空字段：

filter {
    if ![log_msg] {
        mutate {
            add_field => {"log_msg" => "no data" }
        }
    }
}

---

您可以添加一个

mutate

过滤器，该过滤器将在不存在空字段时添加空字段：

filter {
    if ![log_msg] {
        mutate {
            add_field => {"log_msg" => "no data" }
        }
    }
}