Logstash CloudFoundry LoggGator的日志存储配置

我在为Cloud Foundry设置Logstash时遇到了一些问题，我所看到的任何来源都指示我进行以下配置

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}
filter {
if [@type] in ["syslog", "relp"] {
  # Parse Cloud Foundry logs from loggregator (syslog)
  # see https://github.com/cloudfoundry/loggregator/blob/master/src/loggregator/sinks/syslogwriter/syslog_writer.go#L156

  grok {
      match => { "syslog_procid" => "\[(?<log_source>[^/\]]+)(?:/(?<log_source_id>[^\]]+))?\]" }
      tag_on_failure => [
          "fail/logsearch-for-cloudfoundry/loggregator/_grokparsefailure"
      ]
  }

  if !("fail/logsearch-for-cloudfoundry/loggregator/_grokparsefailure" in [tags]) {
      #If it looks like JSON, it must be JSON...
      if [syslog_message] =~ /^\s*{".*}\s*$/ {
          json {
              source => "syslog_message"
          }

          # @todo seems like some messages have @timestamp in them? seems ci-specific
          date {
              match => [ "@timestamp", "ISO8601" ]
          }
      } else {
          mutate {
              add_field => [ "message", "%{syslog_message}" ]
          } 
          if [message] == "-" {
              mutate {
                  remove_field => "message"
              } 
          }
      }

      mutate {
          rename => [ "syslog_program", "@source.app_id" ]
      }

      mutate {
          add_tag => "cloudfoundry_loggregator"
          remove_field => "syslog_facility"
          remove_field => "syslog_facility_code"
          remove_field => "syslog_message"
          remove_field => "syslog_severity"
          remove_field => "syslog_severity_code"
          remove_field => "syslog5424_ver"
          remove_field => "syslog6587_msglen"
      }
  }

} 
}
output {

  stdout { codec => rubydebug }
}

---

如果某些模式已经存在，可以使用Logstash进行解析，请建议确保您在cloud foundry部署清单中确实使用了

relp

协议。如果[“syslog”，“relp”]中的[@type]正在测试该类型，则过滤器

如果您使用的是普通UDP系统日志或TCP，请从筛选器中删除该子句。您可以在kibana中通过转到设置->对象->搜索并禁用该过滤器或删除
relp
条件来完成此操作
 确保您在cloud foundry部署清单中确实使用了
relp
协议。如果[“syslog”，“relp”]

中的[@type]正在测试该类型，则过滤器

如果您使用的是普通UDP系统日志或TCP，请从筛选器中删除该子句。您可以在kibana中通过转到设置->对象->搜索并禁用该过滤器或删除
relp
条件来完成此操作

2015-08-03T09:51:15.000+00:00 [RTR] OUT mm1-spring-music.example.com - [03/08/2015:09:51:15 +0000] "GET /assets/templates/grid.html HTTP/1.1" 200 1450 "http://mm1-spring-music.example.com/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36" 10.10.125.30:37611 x_forwarded_for:"Xx.XX, 0.0.0.0" vcap_request_id:ae307d85-01c3-433b-487d-92d897dbcf99 response_time:0.002201911 app_id:08be9fc8-c7a3-4613-bf12-1a9c7d98cc27