elasticsearch logstash：从时间戳部分创建指纹

我在基于客户端ip和包含日期+小时的时间戳创建指纹时遇到问题。 我使用的是Logstash7.3.1。这里是我的配置文件的相关部分

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date{ 
    match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
  ...
  ruby{
    code => "
      keydate = Date.parse(event.get('timestamp'))
      event.set('keydate', keydate.strftime('%Y%m%d-%H'))
    "
  }
  fingerprint {
    key => "my_custom_secret"
    method => "SHA256"
    concatenate_sources => "true"
    source => [
      "clientip",
      "keydate"
    ]
  }
}

问题出在“ruby”块中。我尝试了多种方法来计算keydate，但没有一种方法不给我错误。 最后一个（使用此配置文件）是

输入文档

{
      "timestamp" => "19/Sep/2019:00:07:56 +0200",
       "referrer" => "-",
       "@version" => "1",
     "@timestamp" => 2019-09-18T22:07:56.000Z,
             ...
        "request" => "index.php",
           "type" => "apache_access",
       "clientip" => "54.157.XXX.XXX",
           "verb" => "GET",
             ...
           "tags" => [
              [0] "_rubyexception"  # generated by the ruby exception above
           ],
       "response" => "200"
}

{
      "timestamp" => "19/Sep/2019:00:07:56 +0200",
       "referrer" => "-",
       "@version" => "1",
     "@timestamp" => 2019-09-18T22:07:56.000Z,
             ...
        "request" => "index.php",
           "type" => "apache_access",
       "clientip" => "54.157.XXX.XXX",
           "verb" => "GET",
             ...
        "keydate" => "20190919-00", #format : YYYYMMDD-HH
    "fingerprint" => "ab347766ef....1190af",
       "response" => "200"
}

预期产出

{
      "timestamp" => "19/Sep/2019:00:07:56 +0200",
       "referrer" => "-",
       "@version" => "1",
     "@timestamp" => 2019-09-18T22:07:56.000Z,
             ...
        "request" => "index.php",
           "type" => "apache_access",
       "clientip" => "54.157.XXX.XXX",
           "verb" => "GET",
             ...
           "tags" => [
              [0] "_rubyexception"  # generated by the ruby exception above
           ],
       "response" => "200"
}

{
      "timestamp" => "19/Sep/2019:00:07:56 +0200",
       "referrer" => "-",
       "@version" => "1",
     "@timestamp" => 2019-09-18T22:07:56.000Z,
             ...
        "request" => "index.php",
           "type" => "apache_access",
       "clientip" => "54.157.XXX.XXX",
           "verb" => "GET",
             ...
        "keydate" => "20190919-00", #format : YYYYMMDD-HH
    "fingerprint" => "ab347766ef....1190af",
       "response" => "200"
}

---

一如既往，非常感谢您的帮助

我建议删除ruby代码段并使用内置日期过滤器：

您在ruby代码片段中所做的正是日期过滤器所做的——从字段中提取时间戳，并将其重新构建为您想要的格式

---

另一个选项（建议少一点，但也可以）是使用grok来提取时间戳的相关部分，并以不同的方式组合它们。

您可以将此行

keydate=Date.parse（event.get（'timestamp'））

更改为：

keydate=Date.parse（event.get（'timestamp'）。改为
。或者，您可以尝试在不使用
Date.parse
函数的情况下设置事件，如下所示：
event.set（'keydate'，Date.strtime（'timestamp'，'%Y%m%d-%H'））
。