Logstash 如果其中一个多重网格匹配,则防止“grokparsefailure”

Logstash 如果其中一个多重网格匹配,则防止“grokparsefailure”,logstash,logstash-grok,Logstash,Logstash Grok,我有个怪癖: grok { patterns_dir => "/etc/logstash/patterns/" break_on_match => false keep_empty_captures => true match => [ "message", "(%{EXIM_DATE:exim_date} )(%{EXIM_PID:exim_pid} )(%{EXIM_MSGID

我有个怪癖:

grok {
        patterns_dir => "/etc/logstash/patterns/"
        break_on_match => false
        keep_empty_captures => true

        match => [ 
            "message", "(%{EXIM_DATE:exim_date} )(%{EXIM_PID:exim_pid} )(%{EXIM_MSGID:exim_msg_id} )(%{EXIM_FLAGS:exim_flags} )(%{GREEDYDATA})",
            "message", "(%{EXIM_MSGID} )(<= )(%{NOTSPACE:env_sender} )(%{EXIM_REMOTE_HOST} )?(%{EXIM_INTERFACE} )?(%{EXIM_PROTOCOL} )?(X=%{NOTSPACE:tls_info} )?(%{EXIM_MSG_SIZE} )?(%{EXIM_HEADER_ID} )?(%{EXIM_SUBJECT})",
            "message", "(%{EXIM_MSGID} )([=-]> )(%{NOTSPACE:env_rcpt} )(<%{NOTSPACE:env_rcpt_outer}> )?(R=%{NOTSPACE:exim_router} )(T=%{NOTSPACE:exim_transport} )(%{EXIM_REMOTE_HOST} )(X=%{NOTSPACE:tls_info} )?(QT=%{EXIM_QT:exim_qt})",
            "message", "(%{EXIM_DATE:exim_date} )(%{EXIM_PID:exim_pid} )(%{EXIM_MSGID:exim_msg_id} )(Completed )(QT=%{EXIM_QT:exim_qt})",
            "message", "(%{EXIM_DATE:exim_date} )(%{EXIM_PID:exim_pid} )(%{EXIM_MSGID:exim_msg_id} )?(%{EXIM_REMOTE_HOST} )?(%EXIM_INTERFACE} )?(F=<%{NOTSPACE:env_sender}> )?(.+(rejected after DATA|rejected \(but fed to sa-learn\)|rejected [A-Z]+ (or [A-Z]+ %{NOTSPACE}?|<%{NOTSPACE:env_rcpt}>)?): (?<exim_rej_reason>.+))"
        ]
      }
grok{
patterns\u dir=>“/etc/logstash/patterns/”
在匹配时中断匹配=>false
保持为空\u捕获=>true
匹配=>[
“消息”,“({EXIM_日期:EXIM_日期})({EXIM_PID:EXIM_PID})({EXIM_MSGID:EXIM_msg_id})({EXIM_标志:EXIM_标志})({GREEDYDATA})”,
“消息”(({EXIM_MSGID})({NOTSPACE:env_rcpt})(?)(R=%{NOTSPACE:EXIM_router})(T=%{NOTSPACE:EXIM_transport})({EXIM_REMOTE_HOST})(X=%{NOTSPACE:tls_info});(QT=%{EXIM QT:EXIM QT}),
“消息”、“({EXIM_DATE:EXIM_DATE})({EXIM_PID:EXIM_PID})({EXIM_MSGID:EXIM_msg_id})(已完成)(QT=%{EXIM_QT:EXIM_QT})”,
“消息”、({EXIM_日期:EXIM_日期})({EXIM_PID:EXIM_PID})({EXIM_MSGID:EXIM_msg_id})({EXIM_远程_主机})(%EXIM_接口});(F=)?(.+(在数据之后被拒绝\(但被馈送到sa学习\);(被拒绝的[A-Z]+(或[A-Z]+{NOTSPACE
]
}
如果我单独测试grok模式,一切都会如预期的那样工作,但在生产中,如果有多个匹配项,它们就不会工作。结果还可以,我解析了所有内容,但每次都会得到一个
\grokparsefailure
标记,如果5个标记中有一个匹配的话。我如何防止这种情况


删除标记不是我想要的,因为如果没有匹配项,则应添加标记,以便我可以删除消息。

导致失败的原因是您已在匹配项上设置了
中断,它将测试匹配项中的每个条目。这会导致其中一个模式不匹配并设置
\u grokparsefailure
标记

从外观上看,您的模式都是彼此独有的,因此您不需要设置
break\u on\u match
,并且仍然保留功能