elasticsearch 分析多行日志：行+；xml

我试图更好地解释： 我有这个多行日志源。 下面我展示了一个包含3个日志的文件，它以“INFO”开头，以“”结尾：

然后我就能够解析JAVA日志和xml了。 但是用我的过滤器（上面的帖子），logstash无法理解日志的结尾在哪里

输出如下所示：

"message" =><dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:02 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:03 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:01",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:02 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:03 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[..]\n</dialogue>
   "server" => [ [0] "FirstLog" ],
   "duration" => [ [0] "311" 

“message”=>\nFirstLog\n311\n[…]\n\nINFO 05-01-16 08:06:02[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\nSecondLog\n500\n[…]\n\nINFO 05-01-16 08:06:03[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\nHirdlog\n100\n[…]
“级别”=>“信息”，
“时间戳”=>“05-01-16 08:06:01”，
“msg_1”=>“http-nio-8080-exec-8”，
“文件”=>“AbstractServer.java”，
“xmldata”=>\nFirstLog\n311\n[…]\n\nINFO 05-01-16 08:06:02[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\n第二日志\n500\n[…]\n\nINFO 05-01-16 08:06:03[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\n\nHirdlog\n100\n[…]\n
“服务器”=>[[0]“第一日志”]，
“持续时间”=>[[0]“311”

logstash只解析第一个xml日志，不考虑其他两个。 我的最终结果应该是：

{
   "message" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:01",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>
   "server" => [ [0] "FirstLog" ],
   "duration" => [ [0] "311"
}
{
   "message" => <dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:02",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" =><dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>
   "server" => [ [0] "SecondLog" ],
   "duration" => [ [0] "500"
}
{
   "message" => <dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:03",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "server" => [ [0] "ThirdLog" ],
   "duration" => [ [0] "100"
}

{
“消息”=>\nFirstLog\n311\n[…]\n
“级别”=>“信息”，
“时间戳”=>“05-01-16 08:06:01”，
“msg_1”=>“http-nio-8080-exec-8”，
“文件”=>“AbstractServer.java”，
“xmldata”=>\nFirstLog\n311\n[…]\n
“服务器”=>[[0]“第一日志”]，
“持续时间”=>[[0]“311”
}
{
“消息”=>\n第二个日志\n500\n[…]\n
“级别”=>“信息”，
“时间戳”=>“05-01-16 08:06:02”，
“msg_1”=>“http-nio-8080-exec-8”，
“文件”=>“AbstractServer.java”，
“xmldata”=>\n第二个日志\n500\n[…]\n
“服务器”=>[[0]“第二日志”]，
“持续时间”=>[[0]“500”
}
{
“消息”=>\nThirdLog\n100\n[…]\n
“级别”=>“信息”，
“时间戳”=>“05-01-16 08:06:03”，
“msg_1”=>“http-nio-8080-exec-8”，
“文件”=>“AbstractServer.java”，
“xmldata”=>\nThirdLog\n100\n[…]\n
“服务器”=>[[0]“第三日志”]，
“持续时间”=>[[0]“100”
}

我希望这是更清楚的，有人有时间给我一些更多的提示

关于

首先，您需要使用多行将所有行连接成一个（根据您的需要，可以使用编解码器或过滤器）

然后，找出日志级别、日期等，并将xml放在自己的字段中

最后，在新的xml字段上使用xml{}过滤器。

首先，您需要使用多行（编解码器或过滤器，取决于您的需要）将所有行连接成一行

然后，找出日志级别、日期等，并将xml放在自己的字段中

---

最后，在您的新xml字段上使用xml{}过滤器。

谢谢，我更改了过滤器。我工作得很好，gork和xml正在解析该行，但现在只有一行。现在我的日志变成：INFO 05-01-16 08:06:01[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\nlocalhost\n311\n[…]\n\n\nINFO 05-01-16 08:06:01[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\nlocalhost\n311\n[…]\n\n[…]我不知道如何定义“结束行”，因为上面我有两个不同的日志。请用新的配置更新您的帖子。谢谢，我更改了我的过滤器。我工作得很好，gork和xml正在解析这一行，但现在我只有一行。现在我的日志变成：INFO 05-01-16 08:06:01[http-nio-8080-exec-8]（AbstractServer.java:454）\n\nlocalhost\n311\n[…]\n\nINFO 05-01-16 08:06:01[http-nio-8080-exec-8]（AbstractServer.java:454）-\n\nlocalhost\n311\n[…]\n\n[…]我不知道如何定义“结束行”，因为上面我有两个不同的日志。请用新配置更新您的帖子。有人有线索吗？有人有线索吗？

"message" =><dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:02 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:03 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:01",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:02 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>\nINFO 05-01-16 08:06:03 [http-nio-8080-exec-8] (AbstractServer.java:454) -\n<dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[..]\n</dialogue>
   "server" => [ [0] "FirstLog" ],
   "duration" => [ [0] "311" 

{
   "message" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:01",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>FirstLog</server>\n<duration>311</duration>\n[...]\n</dialogue>
   "server" => [ [0] "FirstLog" ],
   "duration" => [ [0] "311"
}
{
   "message" => <dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:02",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" =><dialogue>\n<server>SecondLog</server>\n<duration>500</duration>\n[...]\n</dialogue>
   "server" => [ [0] "SecondLog" ],
   "duration" => [ [0] "500"
}
{
   "message" => <dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "level" => "INFO",
   "timestamp" => "05-01-16 08:06:03",
   "msg_1" => "http-nio-8080-exec-8",
   "file" => "AbstractServer.java",
   "xmldata" => <dialogue>\n<server>ThirdLog</server>\n<duration>100</duration>\n[...]\n</dialogue>
   "server" => [ [0] "ThirdLog" ],
   "duration" => [ [0] "100"
}