使用.conf文件logstash获取日志文件中的时间戳
我有一个.conf文件,我想从日志文件中获取时间戳,而不是系统日期。在.conf文件中,我应该做哪些更改,通过logstash馈送到Elasticsearch?日志很好地输入麋鹿堆栈,唯一的问题是它获取系统时间,而不是日志文件中的时间 我的日志行如下使用.conf文件logstash获取日志文件中的时间戳,logstash,elk,Logstash,Elk,我有一个.conf文件,我想从日志文件中获取时间戳,而不是系统日期。在.conf文件中,我应该做哪些更改,通过logstash馈送到Elasticsearch?日志很好地输入麋鹿堆栈,唯一的问题是它获取系统时间,而不是日志文件中的时间 我的日志行如下 input{ file{ path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*" type => "translog"
input{ file{
path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
type => "translog"
start_position => "beginning" } }
filter { grok {
match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\ %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" } }
date{ match => [ "field2" , "yyyy-MM-dd HH:mm:ss SSS" ] } }
output{ elasticsearch{
hosts => "127.0.0.1:9200"
index => "translog" } }
11812211602170772 | 2019-12-19 00:00:00 004 | SPP | U 005206|hsenid217@gmail.com|APP|u 016179 | prov | live | IdeaMart | caas | http | 94771133726 | | | | subs rec charg notify | sms |未知|订户| 16.02 | LKR |[{“currencyCode LKR”,“buyingRate”:1,“sellingRate”:1}]|4.247月日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日/code>
我的.conf文件如下
input{ file{
path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
type => "translog"
start_position => "beginning" } }
filter { grok {
match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\ %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" } }
date{ match => [ "field2" , "yyyy-MM-dd HH:mm:ss SSS" ] } }
output{ elasticsearch{
hosts => "127.0.0.1:9200"
index => "translog" } }
是否有人给出了如何更改.conf文件的示例建议?请尝试下面的答案:
input{
file{
path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
type => "translog"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\
%{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }
}
date {
match => [ "field2", "yyyy-MM-dd HH:mm:ss" ]
target => "@timestamp"
}
}
output{
elasticsearch{
hosts => "127.0.0.1:9200"
index => "translog"
}
}
现在,您的field2
将具有该值2019-12-19 00:00:00
。您希望在字段2中有什么内容?includeformat
too我想获取field2值作为时间戳,并在仪表板中显示它,而不是日志输入的系统/pc时间。我想问的是,为了实现这一点,我应该在.conf文件中做哪些更改?