elasticsearch 使用kafka作为logstash的输入将日志推送到elasticsearch时出错

下面是我的logstash配置文件logstash.conf。LotusStash从卡夫卡主题“我的主题”读取日志，并将其输出到elasticsearch索引“es索引”

input {
 kafka {
       bootstrap_servers =>"kafka1.xxx:9092,kafka2.xxx:9092,kafka3.xxx:9092"
       topics => ["my_topic"]
       codec => "json"
       group_id => "logstashgroup"
 }
}
output {
   elasticsearch {
     hosts => ["es1.myhost:9200","es2.myhost:9200","es3.myhost:9200"]
     user => "user123"
     password => "password"
     index => "es-index"
   }
}
filter {
  json {
   source => "message"
   skip_on_invalid_json => true
 }
}

它在几个月内运行良好，但最近开始抛出以下错误：

[ERROR][logstash.outputs.elasticsearch] An unknown error occurred sending a bulk request to Elasticsearch. We will retry indefinitely {:error_message=>"Could not read from stream: Corrupt GZIP trailer", :error_class=>"Manticore::StreamClosedException"

以及

那么

然后logstash停止了

[ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

---

我不确定是什么问题。我找不到任何相关的东西来解决这个问题。非常感谢您的帮助。

您确定

codec=>json

是正确的吗？不，但是我用

skip\u on\u invalid\u json=>true来忽略无效的json

。它一直工作到最近。您控制生产者代码吗？你最近启用了压缩吗？否则，我不确定我是否理解

损坏的GZIP预告片
想要告诉你什么。也许您应该在logstash中启用调试日志记录，或者使用stdout输出插件查看完整消息的外观
[ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::Kafka codec=><LogStash::Codecs::JSON id=>"json_9a7a9d96-d7be-4292-a3de-67f797d22ab5"

Error: Received exception when fetching the next record from my_topic-1. If needed, please seek past the record to continue consumption.
Exception: Java::OrgApacheKafkaCommon::KafkaException

[ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit