elasticsearch 在logstash中使用多行编解码器组合多个消息字段?
我使用的是Logstash2.4.0elasticsearch 在logstash中使用多行编解码器组合多个消息字段?,elasticsearch,logstash,elasticsearch,Logstash,我使用的是Logstash2.4.0 My output is like this: { "@timestamp" => "2017-05-10T18:14:47.269Z", "message" => "[2017-01-14 10:59:58,591][WARN ][index.search.slowlog.query] [yaswanth] [bank][3] took[50ms], took_millis[50], types[details]
My output is like this:
{
"@timestamp" => "2017-05-10T18:14:47.269Z",
"message" => "[2017-01-14 10:59:58,591][WARN ][index.search.slowlog.query] [yaswanth] [bank][3] took[50ms], took_millis[50], types[details], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}], extra_source[], \r",
"@version" => "1",
"path" => "F:\\logstash-2.4.0\\logstash-2.4.0\\bin\\picaso.txt",
"host" => "yaswanth",
"TIMESTAMP" => "2017-01-14 10:59:58,591",
"LEVEL" => "WARN",
"QUERY" => "index.search.slowlog.query",
"QUERY1" => "yaswanth",
"INDEX-NAME" => "bank",
"SHARD" => "3",
"TOOK" => "50ms",
"TOOKM" => 50,
"types" => "details",
"search_type" => "QUERY_THEN_FETCH",
"total_shards" => "5",
"source_query" => "{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}"
}
{
"@timestamp" => "2017-05-10T18:14:47.270Z",
"message" => "[2017-01-14 10:59:58,591][WARN ][index.search.slowlog.query] [yaswanth] [bank][2] took[50.2ms], took_millis[50], types[details], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}], extra_source[], \r",
"@version" => "1",
"path" => "F:\\logstash-2.4.0\\logstash-2.4.0\\bin\\picaso.txt",
"host" => "yaswanth",
"TIMESTAMP" => "2017-01-14 10:59:58,591",
"LEVEL" => "WARN",
"QUERY" => "index.search.slowlog.query",
"QUERY1" => "yaswanth",
"INDEX-NAME" => "bank",
"SHARD" => "2",
"TOOK" => "50.2ms",
"TOOKM" => 50,
"types" => "details",
"search_type" => "QUERY_THEN_FETCH",
"total_shards" => "5",
"source_query" => "{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}"
}
但我想要的是这样
{
"@timestamp" => "2017-05-10T18:14:47.269Z",
"message" => "[2017-01-14 10:59:58,591][WARN ][index.search.slowlog.query] [yaswanth] [bank][3] took[50ms], took_millis[50], types[details], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}], extra_source[], \r",[2017-01-14 10:59:58,591][WARN ][index.search.slowlog.query] [yaswanth] [bank][2] took[50.2ms], took_millis[50], types[details], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}], extra_source[], \r"
"@version" => "1",
"path" => "F:\\logstash-2.4.0\\logstash-2.4.0\\bin\\picaso.txt",
"host" => "yaswanth",
"TIMESTAMP" => "2017-01-14 10:59:58,591",
"LEVEL" => "WARN",
"QUERY" => "index.search.slowlog.query",
"QUERY1" => "yaswanth",
"INDEX-NAME" => "bank",
"SHARD" => "3",
"TOOK" => "50ms",
"TOOKM" => 50,
"types" => "details",
"search_type" => "QUERY_THEN_FETCH",
"total_shards" => "5",
"source_query" => "{\"sort\":[{\"balance\":{\"order\":\"asc\"}}]}"
}
我想将多个事件中的所有消息字段发送到单个事件以发送电子邮件
上面的配置有什么问题吗?对于这种类型的要求,我必须使用聚合过滤器吗
谢谢您所能做的就是在将事件发送到输出插件之前,在的级别聚合一些事件。给出了一个很好的例子
您可能需要稍微修改一下grok筛选器。您能分享一下您的日志存储配置吗?我编辑了我的问题,意思是您要在输入中一次汇总所有日志并将其作为一个事件发送,对吗?是的。这就是我要说的。我是否必须同时改变输入和输出?我想是的。如果您提供输入文件的一些行,我可能会有额外的帮助。谢谢,但多行编解码器将添加所有消息,但我想应用if条件,因此我使用了聚合筛选器。