elasticsearch 日志存储文本文件输出配置
嘿,我有一个包含许多行的文本文件,每行包含3个由空格分隔的值:elasticsearch 日志存储文本文件输出配置,elasticsearch,logstash,logstash-grok,logstash-configuration,logstash-forwarder,elasticsearch,Logstash,Logstash Grok,Logstash Configuration,Logstash Forwarder,嘿,我有一个包含许多行的文本文件,每行包含3个由空格分隔的值: username email hash username email hash username email hash username email hash username email hash 我尝试使用以下配置使用logstash索引列表: input { file { path => "/path/to/your/file.log"
username email hash
username email hash
username email hash
username email hash
username email hash
我尝试使用以下配置使用logstash索引列表:
input {
file {
path => "/path/to/your/file.log"
start_position => beginning
sincedb_path => "/dev/null"
}
}
filter {
grok {
match => {"message" => "%{WORD:username} %{WORD:email} %{WORD:hash}" }
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
}
我的问题是logstash索引是这样的:
{
"_index": "logstash-2017.06.01",
"_type": "logs",
"_id": "AVxinqK5XRvft8kN7Q6M",
"_version": 1,
"_score": null,
"_source": {
"path": "C:/Users/user/Desktop/user/log.txt",
"@timestamp": "2017-06-01T07:46:22.488Z",
"@version": "1",
"host": "DESKTOP-FNGSJ6C",
"message": "username email password",
"tags": [
"_grokparsefailure"
]
},
"fields": {
"@timestamp": [
1496303182488
]
},
"sort": [
1496303182488
]
}
{
"_index": "logstash-2017.06.01",
"_type": "db",
"_id": "AVxinqK5XRvft8kN7Q6M",
"_version": 1,
"_score": null,
"_source": {
"username": "Marlb0ro",
"email": "Marlb0ro@site.com",
"hash": "123456",
}
我希望它是这样的:
{
"_index": "logstash-2017.06.01",
"_type": "logs",
"_id": "AVxinqK5XRvft8kN7Q6M",
"_version": 1,
"_score": null,
"_source": {
"path": "C:/Users/user/Desktop/user/log.txt",
"@timestamp": "2017-06-01T07:46:22.488Z",
"@version": "1",
"host": "DESKTOP-FNGSJ6C",
"message": "username email password",
"tags": [
"_grokparsefailure"
]
},
"fields": {
"@timestamp": [
1496303182488
]
},
"sort": [
1496303182488
]
}
{
"_index": "logstash-2017.06.01",
"_type": "db",
"_id": "AVxinqK5XRvft8kN7Q6M",
"_version": 1,
"_score": null,
"_source": {
"username": "Marlb0ro",
"email": "Marlb0ro@site.com",
"hash": "123456",
}
我能做些什么来改变它?任何帮助都将被指定当我尝试测试您的搜索时,出现解析错误。由于空格是分隔符,我尝试使用NOTSPACE作为用户名和电子邮件:
%{NOTSPACE:username} %{NOTSPACE:email} %{WORD:hash}
我很确定你的grok解析器不会工作。因为模式词与哈希或电子邮件地址不匹配 您可以在github页面上检查它们的预定义模式
有一个EMAILADDRESS模式,对于哈希,我将使用用户名。对不起,GhostCat,这是一个诚实的错误。我本来应该回答的,但我点击了评论。我把它编辑成:是的,现在看起来好多了。