Events Logstash聚合筛选器,将信息添加到下一行
我正在编写一个Logstash2.4.0配置来浏览HTTP日志。 我们希望在标题字段中传递的端口包含在下面的行字段中。 没有定义特定的结束事件。虽然我也尝试过添加一个结束事件 我当前使用的输入日志文件是:Events Logstash聚合筛选器,将信息添加到下一行,events,logstash,aggregate,multiline,elastic-stack,Events,Logstash,Aggregate,Multiline,Elastic Stack,我正在编写一个Logstash2.4.0配置来浏览HTTP日志。 我们希望在标题字段中传递的端口包含在下面的行字段中。 没有定义特定的结束事件。虽然我也尝试过添加一个结束事件 我当前使用的输入日志文件是: HEADER 9200 LINE 1 2016-10-05 08:39:00 Some log data LINE 2 2016-10-05 08:40:00 Some other log data FOOTER HEADER 9300 LINE 4 2016-11-05 08:39:00 S
HEADER 9200
LINE 1 2016-10-05 08:39:00 Some log data
LINE 2 2016-10-05 08:40:00 Some other log data
FOOTER
HEADER 9300
LINE 4 2016-11-05 08:39:00 Some log data in another log
LINE 5 2016-11-05 08:40:00 Some other log data in another log
FOOTER
我希望有这样的输出:
输出中当前缺少服务器\u端口字段
{"message" => "HEADER 9200",
"@version" => "1",
"@timestamp" => "2016-11-15T11:17:18.425Z",
"path" => "test.log",
"host" => "hostname",
"type" => "event",
"env" => "test",
"port" => 9200,
"tags" => [[0] "Header"] }
{"message" => "LINE 1 2016-10-05 08:39:00 Some log data",
"@version" => "1",
"@timestamp" => "2016-11-15T11:17:20.186Z",
"path" => "test.log",
"host" => "hostname",
"type" => "event",
"env" => "test",
"logMessage" => "1 2016-10-05 08:39:00 Some log data",
"Server_port" => 9200,
"tags" => [[0] "Line"]}
{"message" => "LINE 2 2016-10-05 08:40:00 Some other log data",
"@version" => "1",<
"@timestamp" => "2016-11-15T11:17:20.192Z",
"path" => "test.log",
"host" => "hostname",
"type" => "event",
"env" => "test",
"logMessage" => "2 2016-10-05 08:40:00 Some other log data",
"Server_port" => 9200,
"tags" => [[0] "Line"]}
{"message" => "FOOTER",
"@version" => "1",
"@timestamp" => "2016-11-15T11:17:20.195Z",
"path" => "test.log",
"host" => "hostname",
"type" => "event",
"env" => "test",
"tags" => [[0] "Footer"]}
虽然此配置运行时没有错误,但它不会创建服务器\u端口字段。
我哪里出了问题?在反复尝试之后,我有了一个工作的测试用例。 我已按如下方式更改了配置:
grok {
break_on_match => false
tag_on_failure => []
match => {
"message" => ["^HEADER%{SPACE}%{INT:taskid:int}%{SPACE}%{INT:port:int}"]
}
add_tag => ["Header"]
}
及
并将任务id字段添加到日志中:
HEADER 123 9200
LINE 123 2016-10-05 08:39:00 Some log data
if "Header" in [tags]{
aggregate{
task_id => "%{taskid}"
code => "map['port']=event.get('port')"
map_action => "create"
}
}
elseif "Line" in [tags]{
aggregate{
task_id =>"%{taskid}"
code => "event.set('port',map['port'])"
map_action => "update"
}
}
else if "Footer" in [tags]{
aggregate{
task_id => "%{taskid}"
code => "event.set('port',map['port'])"
map_action => "update"
end_of_task => true
timeout => 120
}
}
HEADER 123 9200
LINE 123 2016-10-05 08:39:00 Some log data