elasticsearch Logstash BeatsParser$InvalidFrameProtocolException

我有filebeat版本1.3.1（amd64） 和logstash 6.3.2

好的，我在filebeat主机上发现ssl错误

Feb 14 17:26:49 dca-cos1-v-ap002 /usr/bin/filebeat[21994]: transport.go:125: SSL client failed to connect with: dial tcp  i/o timeout

因此，我从filebeat.yml中删除tls部分，并在logstash.yml中设置ssl=>false

现在我发现这个错误： 错误如下：

Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]: [2020-02-14T17:38:03,066][INFO ][org.logstash.beats.BeatsHandler] [local: 10.48.1.216:5044, remote: 10.48.1.218:45104] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 1
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]: [2020-02-14T17:38:03,066][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]: io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 1
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:392) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:359) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:236) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]: Caused by: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 1
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at org.logstash.beats.BeatsParser.decode(BeatsParser.java:92) ~[logstash-input-beats-5.0.16.jar:?]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:38:03 PLEUDTLOG01 logstash[8099]:         ... 10 more
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]: [2020-02-14T17:39:03,066][INFO ][org.logstash.beats.BeatsHandler] [local: 10.48.1.216:5044, remote: 10.48.1.218:45106] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 3
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]: [2020-02-14T17:39:03,067][WARN ][io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]: io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 3
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]:         at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
Feb 14 17:39:03 PLEUDTLOG01 logstash[8099]:         at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]

知道我做错了什么吗？

---

这个logstash和filebeat在不久前就开始工作了

我也遇到过这个问题，下面是解决方案

您需要在Logstash pipleline中指定正确的输入

例如，如果您正在使用filebeat，请使用以下命令：

  input {
     beats {
      port => 5044
     } 
   }

如果您将TCP用于springboot或任何实时提要，请使用

input {
  tcp {
   port => 5043
   codec => "json_lines"
   } 
 }

如果同时发送，则配置这两个

input {
   tcp {
    port => 5043
    codec => "json_lines"
   }
   beats {
    port => 5044
    }
  }