Logstash 如何匹配grok中的kafka日志模式

Logstash 如何匹配grok中的kafka日志模式,logstash,logstash-grok,Logstash,Logstash Grok,我想解析卡夫卡日志。下面是一个日志文本示例 [2016-02-01 15:29:02,039] INFO [Replica state machine on controller 0]: Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine) [2016-02-01 15:33:02,457]

我想解析卡夫卡日志。下面是一个日志文本示例

[2016-02-01 15:29:02,039] INFO [Replica state machine on controller 0]: Invoking state change to OnlineReplica for replicas [Topic=elk-test,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
[2016-02-01 15:33:02,457] TRACE [Controller 0]: checking need to trigger partition rebalance (kafka.controller.KafkaController)
[2016-02-01 15:33:02,458] DEBUG [Controller 0]: preferred replicas by broker Map(0 -> Map([elk-test,0] -> List(0))) (kafka.controller.KafkaController)
[2016-02-01 15:33:02,480] DEBUG [Controller 0]: topics not in preferred replica Map() (kafka.controller.KafkaController)
[2016-02-01 15:58:02,447] TRACE [Controller 0]: leader imbalance ratio for broker 0 is 0.000000 (kafka.controller.KafkaController)
我尝试使用grok调试器在grok中编写许多新模式来匹配上面的登录。但是我一直都失败了

我将非常感谢任何能帮助我找到上述日志的grok模式的expect。谢谢

我尝试的模式是:

 \[%{TIMESTAMP_ISO8601:timestamp}\] \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}

我想将控制器0提取为单独的字段

您可以这样做:

\[%{TIMESTAMP_ISO8601:timestamp}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}\[(?<logger>[^\]]+)\]:%{SPACE}%{GREEDYDATA:message}
\[%{TIMESTAMP\u ISO8601:TIMESTAMP}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}\[(?[^\]+)\]:%{SPACE}%{greedyddata:message}

您可以这样做:

\[%{TIMESTAMP_ISO8601:timestamp}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}\[(?<logger>[^\]]+)\]:%{SPACE}%{GREEDYDATA:message}
\[%{TIMESTAMP\u ISO8601:TIMESTAMP}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}\[(?[^\]+)\]:%{SPACE}%{greedyddata:message}