elasticsearch,logstash,jodatime,Datetime,elasticsearch,Logstash,Jodatime" /> elasticsearch,logstash,jodatime,Datetime,elasticsearch,Logstash,Jodatime" />

Datetime ElasticSearch索引问题,无法分析时间戳

Datetime ElasticSearch索引问题,无法分析时间戳,datetime,elasticsearch,logstash,jodatime,Datetime,elasticsearch,Logstash,Jodatime,我是麋鹿新手。 我已经在Elasticsearch中创建了索引 { "logstash": { "aliases": {}, "mappings": { "log": { "dynamic_templates": [ { "message_field": { "path_match": "message", "match_mapping_type":

我是麋鹿新手。 我已经在Elasticsearch中创建了索引

{
  "logstash": {
    "aliases": {},
    "mappings": {
      "log": {
        "dynamic_templates": [
          {
            "message_field": {
              "path_match": "message",
              "match_mapping_type": "string",
              "mapping": {
                "norms": false,
                "type": "text"
              }
            }
          },
          {
            "string_fields": {
              "match": "*",
              "match_mapping_type": "string",
              "mapping": {
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                },
                "norms": false,
                "type": "text"
              }
            }
          }
        ],
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "keyword",
            "include_in_all": false
          },
          "activity": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "beat": {
            "properties": {
              "hostname": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                }
              },
              "name": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                }
              },
              "version": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                }
              }
            }
          },
          "filename": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "host": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "input_type": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "message": {
            "type": "text",
            "norms": false
          },
          "offset": {
            "type": "long"
          },
          "source": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "tags": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "timestamp": {
            "type": "date",
            "include_in_all": false,
            "format": "YYYY-MM-DD HH:mm:ss.SSS"
          },
          "type": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          },
          "user": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword"
              }
            }
          }
        }
      }
    },
    "settings": {
      "index": {
        "creation_date": "1488805244467",
        "number_of_shards": "1",
        "number_of_replicas": "0",
        "uuid": "5ijhh193Tr6y_hxaQrW9kg",
        "version": {
          "created": "5020199"
        },
        "provided_name": "logstash"
      }
    }
  }
}
下面是我的日志存储配置

input{
    beats{
        port=>5044
    }
}filter{
    grok{
        match=>{"message" => "\[%{TIMESTAMP_ISO8601:timestamp}\] ALL AUDIT: User \[%{GREEDYDATA:user}\] is %{GREEDYDATA:activity} \[%{GREEDYDATA:filename}\] for transfer."}
    }
}output{
    elasticsearch{
        hosts=>"localhost:9200"
        index=> "logstash"
    }
样本数据

[2017-03-05 12:37:21.465] ALL AUDIT: User [user1] is opening file [filename1] for transfer.
但当我通过filebeat>logstash>elasticsearch加载文件时 在elasticsearch中,我得到了以下错误

org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp]
Caused by: java.lang.IllegalArgumentException: Invalid format: "2017-03-05T12:36:33.606" is malformed at "12:36:33.606"
    at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[joda-time-2.9.5.jar:2.9.5]

请帮助,我应该配置什么时间戳格式?

在时间戳映射中,您将格式指示为
“格式”:“YYYY-MM-DD HH:MM:ss.SSS”
这里您通过beats发送的格式不相同,请检查:
2017-03-05T12:36:33.606

这就是为什么Elastic会抱怨这种格式。您的格式应该是:
“YYYY-MM-DD'T'HH:MM:ss.SSS”
(注意大写字母T)


有关更多详细信息,请参阅文档:

感谢@asettouf的回复。将日期-时间模式更改为joda格式后问题已解决-“格式”:“yyy-MM-dd HH:MM:ss.SSS”顺便说一句,如果我的答案解决了您的问题,您可能希望使用大复选框将其作为答案接受。它有助于将注意力集中在Stackoverflow上未回答的问题上。