Datetime ElasticSearch索引问题,无法分析时间戳
我是麋鹿新手。 我已经在Elasticsearch中创建了索引Datetime ElasticSearch索引问题,无法分析时间戳,datetime,elasticsearch,logstash,jodatime,Datetime,elasticsearch,Logstash,Jodatime,我是麋鹿新手。 我已经在Elasticsearch中创建了索引 { "logstash": { "aliases": {}, "mappings": { "log": { "dynamic_templates": [ { "message_field": { "path_match": "message", "match_mapping_type":
{
"logstash": {
"aliases": {},
"mappings": {
"log": {
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "keyword",
"include_in_all": false
},
"activity": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"beat": {
"properties": {
"hostname": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"version": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
},
"filename": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"host": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"input_type": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"message": {
"type": "text",
"norms": false
},
"offset": {
"type": "long"
},
"source": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"tags": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"timestamp": {
"type": "date",
"include_in_all": false,
"format": "YYYY-MM-DD HH:mm:ss.SSS"
},
"type": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"user": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1488805244467",
"number_of_shards": "1",
"number_of_replicas": "0",
"uuid": "5ijhh193Tr6y_hxaQrW9kg",
"version": {
"created": "5020199"
},
"provided_name": "logstash"
}
}
}
}
下面是我的日志存储配置
input{
beats{
port=>5044
}
}filter{
grok{
match=>{"message" => "\[%{TIMESTAMP_ISO8601:timestamp}\] ALL AUDIT: User \[%{GREEDYDATA:user}\] is %{GREEDYDATA:activity} \[%{GREEDYDATA:filename}\] for transfer."}
}
}output{
elasticsearch{
hosts=>"localhost:9200"
index=> "logstash"
}
样本数据
[2017-03-05 12:37:21.465] ALL AUDIT: User [user1] is opening file [filename1] for transfer.
但当我通过filebeat>logstash>elasticsearch加载文件时
在elasticsearch中,我得到了以下错误
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp]
Caused by: java.lang.IllegalArgumentException: Invalid format: "2017-03-05T12:36:33.606" is malformed at "12:36:33.606"
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[joda-time-2.9.5.jar:2.9.5]
请帮助,我应该配置什么时间戳格式?在时间戳映射中,您将格式指示为
“格式”:“YYYY-MM-DD HH:MM:ss.SSS”
这里您通过beats发送的格式不相同,请检查:2017-03-05T12:36:33.606
这就是为什么Elastic会抱怨这种格式。您的格式应该是:“YYYY-MM-DD'T'HH:MM:ss.SSS”
(注意大写字母T)
有关更多详细信息,请参阅文档:感谢@asettouf的回复。将日期-时间模式更改为joda格式后问题已解决-“格式”:“yyy-MM-dd HH:MM:ss.SSS”顺便说一句,如果我的答案解决了您的问题,您可能希望使用大复选框将其作为答案接受。它有助于将注意力集中在Stackoverflow上未回答的问题上。