elasticsearch 未分析Grok筛选器,elasticsearch,logstash,elastic-stack,logstash-configuration,elasticsearch,Logstash,Elastic Stack,Logstash Configuration" /> elasticsearch 未分析Grok筛选器,elasticsearch,logstash,elastic-stack,logstash-configuration,elasticsearch,Logstash,Elastic Stack,Logstash Configuration" />

elasticsearch 未分析Grok筛选器

elasticsearch 未分析Grok筛选器,elasticsearch,logstash,elastic-stack,logstash-configuration,elasticsearch,Logstash,Elastic Stack,Logstash Configuration,我正在尝试将Modsecurity-OSSEC日志添加到ELK。日志通过以下浏览器使用filebeat发送到ELK服务器: - paths: - /var/ossec/logs/alerts/alerts.log document_type: ossec-log multiline: pattern: '^\*\*' negate: true match: before max_lines: 20 timeout:

我正在尝试将Modsecurity-OSSEC日志添加到ELK。日志通过以下浏览器使用filebeat发送到ELK服务器:

-
  paths:      
    - /var/ossec/logs/alerts/alerts.log

  document_type: ossec-log
  multiline:
    pattern: '^\*\*'
    negate: true   
    match: before
    max_lines: 20
    timeout: 5s
在ELK服务器上,我在conf.d中创建了一个自定义日志存储配置,其中包含以下内容:

filter {
  if [type] == "ossec-log" {
    grok {
      match => ["message", "(?m)\*\* Alert %{DATA:timestamp_seconds}:%{SPACE}%{WORD}?%{SPACE}\- %{DATA:ossec_group},\n%{YEAR} %{SYSLOGTIMESTAMP:syslog_timestamp}%{SPACE}%{HOSTNAME}\-\>%{DATA:reporting_source}\nRule: %{NONNEGINT:rule_number} \(level %{NONNEGINT:severity}\) \-\> '%{DATA:signature}'\nSrc IP: %{IP:src_ip}\n\[modsecurity\] \[client %{IP:src_ip}\] \[domain %{DATA:FQDN}\] \[%{NONNEGINT:DenyCode}\] \[%{DATA:LogLocation}\]  \[file \"%{DATA:RuleConfig}\"\] \[line \"%{NONNEGINT:LineNumber}\"\] \[id \"%{NONNEGINT:RuleID}\"\] \[rev \"%{NONNEGINT:RevID}\"\] \[msg \"%{DATA:message}\"\] \[severity \"%{DATA:severity_text}\"\]%{GREEDYDATA}"]
    }
  }
}
它应该匹配的数据的结构如下:

** Alert 1471539036.4425320: mail  - modsecurity,access_denied,
2016 Aug 18 18:50:36 myserver->/var/log/httpd/audit_log
Rule: 397989 (level 7) -> 'Atomicorp.com WAF Rules: MSIE 6.0 detected (Disable if you want to allow MSIE 6) '
Src IP: 127.127.127.127
[modsecurity] [client 127.127.127.127] [domain mydomain.org] [403] [/apache/20160818/20160818-1850/20160818-185034-YWhIuJT7g-sAAHaRtlsAAAAv]  [file "/etc/httpd/modsecurity.d/20_asl_useragents.conf"] [line "369"] [id "397989"] [rev "1"] [msg "Atomicorp.com WAF Rules: MSIE 6.0 detected (Disable if you want to allow MSIE 6)"] [severity "WARNING"] Access denied with code 403 (phase 2). Match of "rx (MS Web Services Client Protocol|WormlyBot|webauth@cmcm\\.com)" against "REQUEST_HEADERS:User-Agent" required.
这在grokdebugger中有效,但不知何故,它在ELK中没有被解析。日志项显示在ELK中,类型正确(ossec日志),但它们似乎未被处理。日志条目已列出,但由于整行转储在“message”中,应用了一个名为“beats_input_codec_plain_”的标记,“grokparsefailure”,并且我在grok中定义的字段都不存在

我做错了什么

编辑190816:我通过将匹配括号切换为大括号并将逗号切换为=>(match=>{“message”=>“grok pattern”)。后来我还将filebeat config match指令从“before”切换为“after”