Logstash 如何调试Grok解析失败

我有两个类似的logstash配置。一个正常，另一个不正常，我不知道如何继续

第一个工作配置：

input {
  stdin {
    type => "dec"
  }
}

filter {
  if [type] == "dec" {
    if "SYN/ACK" in [message] or "ACK" in [message] {
      grok {
        match => { "message" => "%{SYSLOGTIMESTAMP:sys_timestamp} %{SYSLOGHOST:sys_hostname} %{DATA:program}: %{NUMBER:ts}:%{GREEDYDATA:reason}:%{IPV4:src_ip:ip}:%{IPV4:dst_ip:ip}:%{POSINT:src_port:int}:%{POSINT:dst_port:int}" }
      }
    } else {
      grok {
        match => { "message" => "%{SYSLOGTIMESTAMP:sys_timestamp} %{SYSLOGHOST:sys_hostname} %{DATA:program}: %{NUMBER:ts}:%{GREEDYDATA:reason}:%{IPV4:src_ip:ip}:%{IPV4:dst_ip:ip}" }
      }
    }
    date {
      timezone => "UTC"
      match => [ "ts", "UNIX_MS" ]
      target => "@timestamp"
    }

    mutate {
      remove_field => [ "message", "sys_timestamp", "sys_hostname" ]
    }

  }
}

output {
  if [type] == "dec" {

    stdout {
      codec => rubydebug
    }
  }
}

Jun 26 10:54:57 rolf-PE-860 sds_lmc:1530035697951:0:6:192.168.10.165:74.125.195.125:::6554:5222:3
Jun 26 10:54:58 rolf-PE-860 sds_lmc:1530035699347:0:17:192.168.10.165:239.255.255.250:::56430:1900:3

第二个错误是解析错误：

input {
    stdin {
    type => "lmc"
}}


filter {
  if [type] == "lmc" {

# Jun 20 17:08:11 rolf-PE-860 sds_lmc: 1529539691952:1:17:::"fe80::14f0:159d:a58d:2802":"ff02::fb":5353:5353:3
# Jun 26 10:54:58 rolf-PE-860 sds_lmc: 1530035699347:0:17:192.168.10.165:239.255.255.250:::56430:1900:3

    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:sys_timestamp} %{SYSLOGHOST:sys_hostname} %{DATA:program}: %{NUMBER:ts}:%{NUMBER:dir}:%{NUMBER:proto}:%{IPV4:src_ip:ip}:%{IPV4:dst_ip:ip}:%{IPV6:src_ip6:ip}:%{IPV6:dst_ip6:ip}:%{POSINT:src_port:int}:%{POSINT:dst_port:int}:%{DATA:act}" }
    }

    date {
      timezone => "UTC"
      match => [ "ts", "UNIX_MS" ]
      target => "@timestamp"
    }


  translate {
    field => "dir"
    destination => "direction"
    dictionary => {
      "0" => "Out"
      "1" => "In"
    }
  }

  translate {
    field => "act"
    destination => "action"
    dictionary => {
      "1" => "Allow"
      "2" => "Block"
      "3" => "Drop"
    }
  }

    mutate {
      remove_field => [ "message", "sys_timestamp", "sys_hostname", "act", "dir" ]
    }
  }
}



output {
  if [type] == "lmc" {
    stdout {
      codec => rubydebug
    }
  }
}

我一直在检查logstash syslog，并试图在[tags]中的“\u grokparsefailure”时获取更多重定向信息。但我没有成功。I laso尝试运行logstash重定向输入和输出，但无法获得足够的信息

工作配置的输入示例：

input {
  stdin {
    type => "dec"
  }
}

filter {
  if [type] == "dec" {
    if "SYN/ACK" in [message] or "ACK" in [message] {
      grok {
        match => { "message" => "%{SYSLOGTIMESTAMP:sys_timestamp} %{SYSLOGHOST:sys_hostname} %{DATA:program}: %{NUMBER:ts}:%{GREEDYDATA:reason}:%{IPV4:src_ip:ip}:%{IPV4:dst_ip:ip}:%{POSINT:src_port:int}:%{POSINT:dst_port:int}" }
      }
    } else {
      grok {
        match => { "message" => "%{SYSLOGTIMESTAMP:sys_timestamp} %{SYSLOGHOST:sys_hostname} %{DATA:program}: %{NUMBER:ts}:%{GREEDYDATA:reason}:%{IPV4:src_ip:ip}:%{IPV4:dst_ip:ip}" }
      }
    }
    date {
      timezone => "UTC"
      match => [ "ts", "UNIX_MS" ]
      target => "@timestamp"
    }

    mutate {
      remove_field => [ "message", "sys_timestamp", "sys_hostname" ]
    }

  }
}

output {
  if [type] == "dec" {

    stdout {
      codec => rubydebug
    }
  }
}

Jun 26 10:54:57 rolf-PE-860 sds_lmc:1530035697951:0:6:192.168.10.165:74.125.195.125:::6554:5222:3
Jun 26 10:54:58 rolf-PE-860 sds_lmc:1530035699347:0:17:192.168.10.165:239.255.255.250:::56430:1900:3

非工作人员：

Jun 21 13:26:21 rolf-PE-860 ips: 1529612781461:ACK/TCP Established:192.168.10.9:192.168.10.165:23:5907
Jun 21 13:26:29 rolf-PE-860 ips: 1529612789554:ARP Response:192.168.10.127:192.168.10.140

以下是配置失败的结果

{
          "tags" => [
        [0] "_grokparsefailure"
    ],
          "host" => "rolf-PE-860",
      "@version" => "1",
          "type" => "lmc",
    "@timestamp" => 2018-06-26T23:41:49.349Z
}
{
          "tags" => [
        [0] "_grokparsefailure"
    ],
          "host" => "rolf-PE-860",
      "@version" => "1",
          "type" => "lmc",
    "@timestamp" => 2018-06-26T23:41:49.355Z
}

如果有人能帮我提供一些如何进行的建议，我将不胜感激

---

感谢调试grok筛选器，这里有一些有用的资源：a和感谢。我先试过Grokconstructor，但我会再试一次，因为我现在的沮丧程度更低了。@baudsp-谢谢。我用了更多的时间，学习了更多关于如何使用这个工具的知识。这一点耐心地解决了一系列问题，从一个放错位置的“”开始。下次我会更容易解决问题。