Logstash不会处理跨多行的日志

我正在尝试解析一些本地日志文件，我正在windows计算机上运行ELK堆栈。下面是我试图解析的日志示例

2015-12-10 13:50:25,487 [http-nio-8080-exec-26] INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- This Message is OK
2015-12-10 13:50:26,487 [http-nio-8080-exec-26] INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- Journe
y road update: <rows>
     <row adi="D" date="2015-12-10" garage="TOP">
          <codeNum order="1">TP</codeNum>
          <number order="1">1001</number>
          <journeystatus code="RT">OnRoute</journeystatus>
     </row>
</rows>

当我跑的时候

bin\logstash agent -f \ELK-Stack\logstash\conf\01_input.conf

在CMD提示符中，返回的内容如下

io/console not supported; tty will not be manipulated
Default settings used: Filter workers: 4
Logstash startup completed
{
            "message" => "     <row adi=\"D\" date=\"2015-12-10\" garage=\"TOP\"
>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.268Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "          <codeNum order=\"1\">TP</codeNum>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.268Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "          <number order=\"1\">1001</number>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.268Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "          <journeystatus code=\"RT\">OnRoute</journeys
tatus>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.278Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "     </row>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.278Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "y road update: <rows>\r",
           "@version" => "1",
         "@timestamp" => "2015-12-11T12:49:34.268Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
               "tags" => [
        [0] "_grokparsefailure"
    ],
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "2015-12-10 13:50:25,487 [http-nio-8080-exec-26] INFO
a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- Journe\r",
           "@version" => "1",
         "@timestamp" => "2015-12-10T13:50:25.487Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
       "logTimestamp" => "2015-12-10;13:50:25.487",
           "threadId" => "http-nio-8080-exec-26",
           "logLevel" => "INFO",
          "JavaClass" => "a.b.c.v1.myTestClass",
      "TransactionID" => "[abcde-1234-12345-b425-12ad]",
         "LogMessage" => "- Journe\r",
    "debug-timestamp" => "timestampMatched"
}
{
            "message" => "</rows>2015-12-10 13:50:25,487 [http-nio-8080-exec-26]
 INFO  a.b.c.v1.myTestClass [abcde-1234-12345-b425-12ad]- This Message is OK\r",

           "@version" => "1",
         "@timestamp" => "2015-12-10T13:50:25.487Z",
               "host" => "GMAN",
               "path" => "C:/data/sampleLogs/temp.log",
               "type" => "testlog",
       "logTimestamp" => "2015-12-10;13:50:25.487",
           "threadId" => "http-nio-8080-exec-26",
           "logLevel" => "INFO",
          "JavaClass" => "a.b.c.v1.myTestClass",
      "TransactionID" => "[abcde-1234-12345-b425-12ad]",
         "LogMessage" => "- This Message is OK\r",
    "debug-timestamp" => "timestampMatched"
}

但这并没有帮助，只是不断给我一个错误信息，如

Error: Cannot use more than 1 filter worker because the following plugins don't
work with more than one worker: multiline
You may be interested in the '--configtest' flag which you can
use to validate logstash's configuration before you choose
to restart a running system.

因此，我尝试按照建议运行

--configtest

，并显示一条新的错误消息

Error: Cannot use more than 1 filter worker because the following plugins don't
work with more than one worker: multiline

有人能帮我解决这个问题，让logstash处理多行吗

非常感谢你的帮助

更新

正如@Alain Collins建议使用codec with，下面是我的配置的输入

input {
    file {
        path => "C:/data/sampleLogs/mulline.log"
        codec => multiline {
            # Grok pattern names are valid! :)
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
        }
        type => "testlog"
        start_position => "beginning"
    }
}

---

G

您找到了正确的解决方案-多行。这些线路需要连接成一个事件

正如您所发现的，多行筛选器不是线程安全的，因此您只能在该日志库中运行一个工作线程

有一种方法可能对你有用。它将这些行组合为输入{}阶段的一部分，并将一个事件传递给过滤器{}阶段

请注意，您可以将logstash模式与多行一起使用，因此“^%{YEAR}”比“^201”更好

---

最后，请注意，这是logstash forwarder的替代品。他们说计划提供客户端多行支持，因此消息将作为一个事件从客户端发送，而不必由logstash重新组装。

感谢@Alain Collins您的帮助，帮助我实现了我想要做的事情。我已经更新了我的问题，并将其标记为已回答。

Error: Cannot use more than 1 filter worker because the following plugins don't
work with more than one worker: multiline

input {
    file {
        path => "C:/data/sampleLogs/mulline.log"
        codec => multiline {
            # Grok pattern names are valid! :)
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
        }
        type => "testlog"
        start_position => "beginning"
    }
}