Logging 配置Logstash以解码其自身的事件格式JSON

Logging 配置Logstash以解码其自身的事件格式JSON,logging,logstash,kibana,logstash-forwarder,logstash-logback-encoder,Logging,Logstash,Kibana,Logstash Forwarder,Logstash Logback Encoder,我有一个Web应用程序的java日志文件,它是使用SLF4J、Logback和在logstash 1.4.2中使用创建的。虽然各种配置都成功地从日志中检索到了数据,但实际上都没有返回正确的json。根据我阅读的每个指南,以下配置应该可以工作,但不能 原木样品 {"@timestamp":"2015-02-04T00:03:43.178+00:00","@version":1,"message":"No token was found, creating new token.","logger_n

我有一个Web应用程序的java日志文件,它是使用SLF4J、Logback和在logstash 1.4.2中使用创建的。虽然各种配置都成功地从日志中检索到了数据,但实际上都没有返回正确的json。根据我阅读的每个指南,以下配置应该可以工作,但不能

原木样品

{"@timestamp":"2015-02-04T00:03:43.178+00:00","@version":1,"message":"No token was found, creating new token.","logger_name":"com.company.ws.service.AuthService","thread_name":"ajp-nio-8009-exec-10","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.199+00:00","@version":1,"message":"5f8aaebd-4274-4f00-a2eb-7b2350231ef2","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-1","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.199+00:00","@version":1,"message":"36","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-1","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.218+00:00","@version":1,"message":"5f8aaebd-4274-4f00-a2eb-7b2350231ef2","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-3","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.218+00:00","@version":1,"message":"36","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-3","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.218+00:00","@version":1,"message":"135a2411-ac96-492b-94e9-df6b65974f9f","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-3","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.218+00:00","@version":1,"message":"36","logger_name":"com.company.jaxrs.provider.ParamTest","thread_name":"ajp-nio-8009-exec-3","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
{"@timestamp":"2015-02-04T00:03:43.219+00:00","@version":1,"message":"is string","logger_name":"com.company.jaxrs.parameter.RestParameterFactory","thread_name":"ajp-nio-8009-exec-3","level":"INFO","level_value":20000,"HOSTNAME":"development.company.com"}
/etc/logstash/conf.d/01-lumberjack-input.conf /etc/logstash/conf.d/10-syslog.conf /etc/logstash/conf.d/30-lumberjack-output.conf /etc/物流转运商(其他机器)
我在Kibana获得的最好回报(如果有回报的话)如下:

{
  "_index": "logstash-2015.02.04",
  "_type": "json",
  "_id": "8l1rDYTZSceBCklFxAuvAg",
  "_score": null,
  "_source": {
    "message": "{\"@timestamp\":\"2015-02-04T06:03:18.794+00:00\",\"@version\":1,\"message\":\"Attribute Count 1\",\"logger_name\":\"com.company.ws.service.ReportSearchService\",\"thread_name\":\"ajp-nio-8009-exec-1\",\"level\":\"INFO\",\"level_value\":20000,\"HOSTNAME\":\"development.company.com\"}",
    "@version": "1",
    "@timestamp": "2015-02-04T06:13:10.685Z",
    "type": "json",
    "file": "/company/apache-tomcat-8.0.9/logs/vhost1.log",
    "host": "development.company.com",
    "offset": "4907321"
  },
  "sort": [
    1423030390685,
    1423030390685
  ]
}
显然,json转换逻辑没有正常工作,那么我缺少什么呢


ELK堆栈是使用配置的。

这看起来非常可疑:

else if [type] == "json" {

      source => "message"

}
如果这真的是你的配置文件中的内容,我不明白为什么Logstash不抱怨它。它应该是这样的:

else if [type] == "json" {
  json {
    source => "message"
  }
}
或者,如果通过lumberjack协议接收到的所有消息都是JSON消息,则可以使用JSON编解码器进行lumberjack输入

{
  "network": {
    "servers": [ "utility.company.com:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
  },
  "files": [
    {
      "paths": ["/company/apache-tomcat-8.0.9/logs/vhost1.log"],
      "fields": { "type": "json"  }

    }
   ]
}
{
  "_index": "logstash-2015.02.04",
  "_type": "json",
  "_id": "8l1rDYTZSceBCklFxAuvAg",
  "_score": null,
  "_source": {
    "message": "{\"@timestamp\":\"2015-02-04T06:03:18.794+00:00\",\"@version\":1,\"message\":\"Attribute Count 1\",\"logger_name\":\"com.company.ws.service.ReportSearchService\",\"thread_name\":\"ajp-nio-8009-exec-1\",\"level\":\"INFO\",\"level_value\":20000,\"HOSTNAME\":\"development.company.com\"}",
    "@version": "1",
    "@timestamp": "2015-02-04T06:13:10.685Z",
    "type": "json",
    "file": "/company/apache-tomcat-8.0.9/logs/vhost1.log",
    "host": "development.company.com",
    "offset": "4907321"
  },
  "sort": [
    1423030390685,
    1423030390685
  ]
}
else if [type] == "json" {

      source => "message"

}
else if [type] == "json" {
  json {
    source => "message"
  }
}