elasticsearch 如何解决logstash中的日期解析错误?
我有以下日志存储配置:elasticsearch 如何解决logstash中的日期解析错误?,elasticsearch,logstash,logstash-configuration,elasticsearch,Logstash,Logstash Configuration,我有以下日志存储配置: input { file{ path => ["C:/Users/MISHAL/Desktop/ELK_Files/rm/evsb.json"] type => "json" start_position => "beginning" } } filter { json { source => "message" } mutate { conve
input {
file{
path => ["C:/Users/MISHAL/Desktop/ELK_Files/rm/evsb.json"]
type => "json"
start_position => "beginning"
}
}
filter {
json {
source => "message"
}
mutate {
convert => [ "increasedFare", "float"]
convert => ["enq", "float"]
convert => ["bkd", "float"]
}
date{
match => [ "date" , "YYYY-MM-dd HH:mm:ss" ]
target => "@timestamp"
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => "localhost"
index => "zsx"
}
}
这是json数据jt.json:
[{"id":1,"date":"2015-11-11 23:00:00","enq":"105","bkd":"9","increasedFare":"0"}, {"id":2,"date":"2015-11-15 23:00:00","eng":"55","bkd":"2","increasedFare":"0"}, {"id":3,"date":"2015-11-20 23:00:00","enq":"105","bkd":"9","increasedFare":"0"}, {"id":4,"date":"2015-11-25 23:00:00","eng":"55","bkd":"2","increasedFare":"0"}]
尝试在logstash中运行此命令,但是我无法解析日期或在时间戳中获取日期。
以下是我收到的警告消息:
Failed parsing date from field {:field=>"[date]", :value=>"%{[date]}", :exception=>"Invalid format: \"%{[date]}\"", :config_parsers=>"YYYY-MM-dd HH:mm:ss", :config_locale=>"default=en_IN", :level=>:warn}
以下是标准
Logstash startup completed
{
"message" => "{\"id\":2,\"date\":\"2015-09-15 23:00:00\",\"enq\":\"34\",\"bkd\":\"2\",\"increasedFare\":\"0\"}\r",
"@version" => "1",
"@timestamp" => "2015-09-15T17:30:00.000Z",
"host" => "TCHWNG",
"path" => "C:/Users/MISHAL/Desktop/ELK_Files/jsonTest/jt.json",
"type" => "json",
"id" => 2,
"date" => "2015-09-15 23:00:00",
"enq" => 34.0,
"bkd" => 2.0,
"increasedFare" => 0.0
}
{
"message" => "{\"id\":3,\"date\":\"2015-09-20 23:00:00\",\"enq\":\"22\",\"bkd\":\"9\",\"increasedFare\":\"0\"}\r",
"@version" => "1",
"@timestamp" => "2015-09-20T17:30:00.000Z",
"host" => "TCHWNG",
"path" => "C:/Users/MISHAL/Desktop/ELK_Files/jsonTest/jt.json",
"type" => "json",
"id" => 3,
"date" => "2015-09-20 23:00:00",
"enq" => 22.0,
"bkd" => 9.0,
"increasedFare" => 0.0
}
{
"message" => "{\"id\":4,\"date\":\"2015-09-25 23:00:00\",\"enq\":\"66\",\"bkd\":\"2\",\"increasedFare\":\"0\"}\r",
"@version" => "1",
"@timestamp" => "2015-09-25T17:30:00.000Z",
"host" => "TCHWNG",
"path" => "C:/Users/MISHAL/Desktop/ELK_Files/jsonTest/jt.json",
"type" => "json",
"id" => 4,
"date" => "2015-09-25 23:00:00",
"enq" => 66.0,
"bkd" => 2.0,
"increasedFare" => 0.0
}
两天来我一直在努力解决这个问题,尝试了各种各样的方法,但我无法解决这个问题。请告诉我我在这里做错了什么。你的照片上的记录是什么样子的?如果[date]在json中,为什么要使用add_字段?您是否尝试过只使用“date”(而不是“[date]”的日期过滤器?我更改了配置文件,还必须更改json数据。在json文件中,我将每条记录添加为单独的行,并删除了每条记录之间的“,”。这解决了这个问题,但我觉得很奇怪,我希望logstash能够处理标准json格式。我现在已经把我的问题包括在内了。