<img src="//i.stack.imgur.com/RUiNP.png" height="16" width="18" alt="" class="sponsor tag img">elasticsearch 多行分析模式_<img Src="//i.stack.imgur.com/RUiNP.png" Height="16" Width="18" Alt="" Class="sponsor Tag Img">elasticsearch_Logstash_Elastic Stack_Filebeat

elasticsearch 多行分析模式

logstash

elasticsearch 多行分析模式,elasticsearch,logstash,elastic-stack,filebeat,elasticsearch,Logstash,Elastic Stack,Filebeat,我想在ELK stack 6.3.2版本中解析一个标准JAVA异常，如下所示： 2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on ser

我想在ELK stack 6.3.2版本中解析一个标准JAVA异常，如下所示：

2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

input {

beats {
    port=>5044
        codec => multiline {
               pattern => "^\s"
              what => "previous"
}
}

My filebeat.yml具有以下配置：

filebeat.inputs:

- type: log
  enabled: true
  paths:
    - C:\logs\test.log
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

我的logstash.conf输入如下所示：

2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

input {

beats {
    port=>5044
        codec => multiline {
               pattern => "^\s"
              what => "previous"
}
}

但是logstash说未能解析该模式，事实上，它异常崩溃。如果仅删除

编解码器

配置，则会解析异常的第一行。我亦曾在会议上提出同样的问题，但没有回应

您需要更改：

多行。求反

至

真

。我不确定你想通过这种模式实现什么，但似乎你应该这样做：

multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'

此外，您不需要在日志存储中使用多行-只需简单地使用：

input {
    beats {
        port => 5044            
    }
}

在夏季，为了捕获所有日志，我将您的Filebeat配置更改为：

- type: log
  enabled: true
  paths:
    - C:\logs\test.log
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

一点解释：当我们选择

negate:true

和

match:after

时，我们告诉FileBeat：

与图案不匹配的连续行将追加到前一行不匹配

换句话说，它告诉FileBeat获取以给定模式开始的每一行，并在该模式再次出现在新行的开头时停止。
对于此模式

^[0-9]{4}-[0-9]{2}-[0-9]{2}

，如果您得到这两个异常：

2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

2018-09-04 05:30:00.000 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
    at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
    at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) ...

它会将每个异常捕获为不同的条目日志。如果您记录了更多的内容，并且希望filebeat只捕获错误，那么这是另一回事。在我们的程序中，我们收集所有信息并按严重程度（即错误、信息、警告等）进行查询。