Amazon s3 Logstash S3输入-筛选日志类型

我正在用麋鹿堆（Elasticsearch、Logstash和Kibana）集中日志。它工作得很好，但是

我的S3存储桶中有几种类型的日志：

• elasticbeanstalk-访问日志
• 错误日志
• tomcat7访问日志
• stacktrace日志

我正在日志存储配置文件中使用S3输入插件：

input {
 s3 {
    secret_access_key => "..."
    access_key_id => "..."
    region => "eu-central-1"
    bucket => "bucket_name"
    prefix => "resources/environments/logs/publish"
    codec => "plain"
  }
}

我正在使用一些过滤器插件：

filter {
 if [type] ==  "access" { 
    mutate { replace => { type =>  "apache_access" } } 
    grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } 
    date { match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] } 
 } else {
    multiline {
        #type => "all" # no type means for all inputs
        pattern => "(^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)"
        what => "previous" 
    }

    grok  {
        match => [ "message", "(?m)%{TIMESTAMP_ISO8601:timestamp} \[%{HOSTNAME:thread}\] %{LOGLEVEL:severity} %{GREEDYDATA:message}" ]
        overwrite => [ "message" ]
    }

    date {
        match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
  }
}

问题：有4种类型。如何使用“如果”来过滤日志。我使用“”来测试grok过滤器，它适用于1种类型的日志

解决方案应该是这样的：

if [type] ==  "access" { 
   #my grok filter 
} else if [type] == "stacktrace" {
   #my grok filter
} else if [type] == "tomcat7" {
  #my grok filter
} ...

Tomcat Cataline输出日志：

    2016-04-07 15:27:28,459 [http-bio-8080-exec-33] ERROR v1.PaymentTxController  - Cannot get property 'attrs' on null object
java.lang.NullPointerException: Cannot get property 'attrs' on null object
    at com.b2boost.payment.provider.paybox.PayboxPaymentProviderService.createSubscriptionAndPay(PayboxPaymentProviderService.groovy:206)
    at com.b2boost.payment.provider.paybox.PayboxPaymentProviderService$__tt__pay_closure9.doCall(PayboxPaymentProviderService.groovy:82)
    at com.b2boost.commons.error.AppError.safe(AppError.groovy:53)
    at com.b2boost.commons.error.AppError.safe(AppError.groovy:60)
    at com.b2boost.payment.provider.paybox.PayboxPaymentProviderService.$tt__pay(PayboxPaymentProviderService.groovy:73)
    at com.b2boost.payment.PaymentService$__tt__pay_closure8.doCall(PaymentService.groovy:52)
    at com.b2boost.commons.error.AppError.safeWithEither(AppError.groovy:70)
    at com.b2boost.commons.error.AppError.safeWithEither(AppError.groovy:64)
    at com.b2boost.payment.PaymentService.$tt__pay(PaymentService.groovy:43)
    at com.b2boost.users.api.v1.PaymentTxController$_save_closure1.doCall(PaymentTxController.groovy:49)
    at com.b2boost.users.api.v1.BaseController.documentWithAuthorization(BaseController.groovy:101)
    at com.b2boost.users.api.v1.PaymentTxController.save(PaymentTxController.groovy:45)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:177)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at com.odobo.grails.plugin.springsecurity.rest.RestTokenValidationFilter.processFilterChain(RestTokenValidationFilter.groovy:99)
    at com.odobo.grails.plugin.springsecurity.rest.RestTokenValidationFilter.doFilter(RestTokenValidationFilter.groovy:66)
    at grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.java:53)
    at com.odobo.grails.plugin.springsecurity.rest.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:108)
    at grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.java:82)
    at com.odobo.grails.plugin.springsecurity.rest.RestLogoutFilter.doFilter(RestLogoutFilter.groovy:63)
    at com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

错误日志：

[Tue Apr 12 10:01:01 2016] [notice] Apache/2.2.29 (Unix) DAV/2 configured -- resuming normal operations

堆栈跟踪日志

2015-11-13 16:02:28,524 [MonitoringThread-118] ERROR StackTrace  - Full Stack Trace:
com.notnoop.exceptions.ApnsDeliveryErrorException: Failed to deliver notification with error code 8
    at com.notnoop.apns.internal.ApnsConnectionImpl$2.run(ApnsConnectionImpl.java:189)
    at java.lang.Thread.run(Thread.java:745)

---

一旦您有了一个字段，您就可以在条件语句中使用它，如您所示。尝试时会发生什么？它无法识别[type]条件。。。可能是因为没有“日志”类型？我不知道你在什么地方设置“类型”吗？stdout{}输出中的事件是什么样子的？我添加了一些日志示例