使用.conf文件logstash获取日志文件中的时间戳

使用.conf文件logstash获取日志文件中的时间戳,logstash,elk,Logstash,Elk,我有一个.conf文件,我想从日志文件中获取时间戳,而不是系统日期。在.conf文件中,我应该做哪些更改,通过logstash馈送到Elasticsearch?日志很好地输入麋鹿堆栈,唯一的问题是它获取系统时间,而不是日志文件中的时间 我的日志行如下 input{ file{ path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*" type => "translog"

我有一个.conf文件,我想从日志文件中获取时间戳,而不是系统日期。在.conf文件中,我应该做哪些更改,通过logstash馈送到Elasticsearch?日志很好地输入麋鹿堆栈,唯一的问题是它获取系统时间,而不是日志文件中的时间

我的日志行如下

input{  file{
        path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
        type => "translog"
        start_position => "beginning"   } }

filter {     grok {
            match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\ %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }     }

    date{       match => [ "field2" , "yyyy-MM-dd HH:mm:ss SSS" ]       } }

output{     elasticsearch{
        hosts => "127.0.0.1:9200"
        index => "translog"     } }
11812211602170772 | 2019-12-19 00:00:00 004 | SPP | U 005206|hsenid217@gmail.com|APP|u 016179 | prov | live | IdeaMart | caas | http | 94771133726 | | | | subs rec charg notify | sms |未知|订户| 16.02 | LKR |[{“currencyCode LKR”,“buyingRate”:1,“sellingRate”:1}]|4.247月日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日日/code>

我的.conf文件如下

input{  file{
        path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
        type => "translog"
        start_position => "beginning"   } }

filter {     grok {
            match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\ %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }     }

    date{       match => [ "field2" , "yyyy-MM-dd HH:mm:ss SSS" ]       } }

output{     elasticsearch{
        hosts => "127.0.0.1:9200"
        index => "translog"     } }

是否有人给出了如何更改.conf文件的示例建议?

请尝试下面的答案:

input{
        file{
            path => "/home/rehan/Projects/SIEM/Splunk/test3/sdp-server/*"
            type => "translog"
            start_position => "beginning"
        }
    }

    filter {
         grok {
                match => { "message" => "%{NUMBER:field1}\|%{TIMESTAMP_ISO8601:field2} %{NUMBER:field3}\|%{WORD:field4}\|*\|%{WORD:field5}\@%{WORD:field6}\.com\|%{WORD:field7}\|%{WORD:field8}\|%{WORD:field9}\|%{WORD:field10}\|\|%{WORD:field11}\|%{WORD:field12}\|%{NUMBER:field13}\|\|\|\|%{WORD:field14}\-%{WORD:field15}\-%{WORD:field16}\-%{WORD:field17}\|%{WORD:field18}\|%{WORD:field19}\|%{WORD:field20}\|%{NUMBER:field21}\|%{WORD:field22}\|\[\{\"%{WORD:field23}\"\:\"%{WORD:field24}\"\,\ \"%{WORD:field25}\"\:\
     %{NUMBER:field26}\, \"%{WORD:field27}\"\: %{NUMBER:field28}\}]\|%{WORD:field29}\|%{NUMBER:field30}\|\|%{WORD:field31}\|%{WORD:field32}\|%{WORD:field33}\|%{WORD:field34}\|%{WORD:field35}\|%{NUMBER:field36}\|%{WORD:field37}\|\|%{WORD:field38}\|%{NUMBER:field39}\|\|\|\|%{WORD:field40}\|%{WORD:field41}\|%{NUMBER:field42}\|%{WORD:field43}\|%{NUMBER:field43}\|\|\|\|\|\|\|\|\|\|\|%{WORD:field44}" }
        }
        date {
            match => [ "field2", "yyyy-MM-dd HH:mm:ss" ]
            target => "@timestamp"
        }
    }

    output{ 
        elasticsearch{
            hosts => "127.0.0.1:9200"
            index => "translog"
        }
    }

现在,您的
field2
将具有该值
2019-12-19 00:00:00
。您希望在
字段2中有什么内容?include
format
too我想获取field2值作为时间戳,并在仪表板中显示它,而不是日志输入的系统/pc时间。我想问的是,为了实现这一点,我应该在.conf文件中做哪些更改?