elasticsearch 日志存储文本文件输出配置,elasticsearch,logstash,logstash-grok,logstash-configuration,logstash-forwarder,elasticsearch,Logstash,Logstash Grok,Logstash Configuration,Logstash Forwarder" /> elasticsearch 日志存储文本文件输出配置,elasticsearch,logstash,logstash-grok,logstash-configuration,logstash-forwarder,elasticsearch,Logstash,Logstash Grok,Logstash Configuration,Logstash Forwarder" />

elasticsearch 日志存储文本文件输出配置

elasticsearch 日志存储文本文件输出配置,elasticsearch,logstash,logstash-grok,logstash-configuration,logstash-forwarder,elasticsearch,Logstash,Logstash Grok,Logstash Configuration,Logstash Forwarder,嘿,我有一个包含许多行的文本文件,每行包含3个由空格分隔的值: username email hash username email hash username email hash username email hash username email hash 我尝试使用以下配置使用logstash索引列表: input { file { path => "/path/to/your/file.log"

嘿,我有一个包含许多行的文本文件,每行包含3个由空格分隔的值:

username email hash
username email hash
username email hash
username email hash
username email hash
我尝试使用以下配置使用logstash索引列表:

    input {
        file {
                path => "/path/to/your/file.log"
                start_position => beginning
                sincedb_path => "/dev/null"
        }
}
filter {
        grok {
                match => {"message" => "%{WORD:username} %{WORD:email} %{WORD:hash}" }
        }
}
output {
        elasticsearch {
                hosts => ["localhost:9200"]
        }
}
我的问题是logstash索引是这样的:

        {
  "_index": "logstash-2017.06.01",
  "_type": "logs",
  "_id": "AVxinqK5XRvft8kN7Q6M",
  "_version": 1,
  "_score": null,
  "_source": {
    "path": "C:/Users/user/Desktop/user/log.txt",
    "@timestamp": "2017-06-01T07:46:22.488Z",
    "@version": "1",
    "host": "DESKTOP-FNGSJ6C",
    "message": "username email password",
    "tags": [
      "_grokparsefailure"
    ]
  },
  "fields": {
    "@timestamp": [
      1496303182488
    ]
  },
  "sort": [
    1496303182488
  ]
}
        {
  "_index": "logstash-2017.06.01",
  "_type": "db",
  "_id": "AVxinqK5XRvft8kN7Q6M",
  "_version": 1,
  "_score": null,
  "_source": {
    "username": "Marlb0ro",
    "email": "Marlb0ro@site.com",
    "hash": "123456",
}
我希望它是这样的:

        {
  "_index": "logstash-2017.06.01",
  "_type": "logs",
  "_id": "AVxinqK5XRvft8kN7Q6M",
  "_version": 1,
  "_score": null,
  "_source": {
    "path": "C:/Users/user/Desktop/user/log.txt",
    "@timestamp": "2017-06-01T07:46:22.488Z",
    "@version": "1",
    "host": "DESKTOP-FNGSJ6C",
    "message": "username email password",
    "tags": [
      "_grokparsefailure"
    ]
  },
  "fields": {
    "@timestamp": [
      1496303182488
    ]
  },
  "sort": [
    1496303182488
  ]
}
        {
  "_index": "logstash-2017.06.01",
  "_type": "db",
  "_id": "AVxinqK5XRvft8kN7Q6M",
  "_version": 1,
  "_score": null,
  "_source": {
    "username": "Marlb0ro",
    "email": "Marlb0ro@site.com",
    "hash": "123456",
}

我能做些什么来改变它?任何帮助都将被指定

当我尝试测试您的搜索时,出现解析错误。由于空格是分隔符,我尝试使用NOTSPACE作为用户名和电子邮件:

%{NOTSPACE:username} %{NOTSPACE:email} %{WORD:hash}

我很确定你的grok解析器不会工作。因为模式词与哈希或电子邮件地址不匹配

您可以在github页面上检查它们的预定义模式


有一个EMAILADDRESS模式,对于哈希,我将使用用户名。

对不起,GhostCat,这是一个诚实的错误。我本来应该回答的,但我点击了评论。我把它编辑成:是的,现在看起来好多了。