Apache kafka Kafka字节在监视中被拒绝

Apache kafka Kafka字节在监视中被拒绝,apache-kafka,jmx,Apache Kafka,Jmx,我们安装了8个节点的kafka群集和kafka管理器 我们正在通过new relic进行监控 new Relic和kafka manager都报告kafka拒绝字节。我找不到原因 在代理日志中没有错误行 jmsbean-JMX/kafka.server/BrokerTopicMetrics/BytesRejectedPerSec/OneMinute 卡夫卡形态- auto.create.topics.enable=false auto.leader.rebalance.enable=true

我们安装了8个节点的kafka群集和kafka管理器

我们正在通过new relic进行监控

new Relic和kafka manager都报告kafka拒绝字节。我找不到原因

在代理日志中没有错误行

jmsbean-JMX/kafka.server/BrokerTopicMetrics/BytesRejectedPerSec/OneMinute

卡夫卡形态-

auto.create.topics.enable=false
auto.leader.rebalance.enable=true
broker.id=180
controlled.shutdown.enable=true
controlled.shutdown.max.retries=3
default.replication.factor=1
delete.topic.enable=true
kafka.http.metrics.host=0.0.0.0
kafka.http.metrics.port=24042
kafka.log4j.dir=/logs/kafka
kerberos.auth.enable=false
leader.imbalance.check.interval.seconds=300
leader.imbalance.per.broker.percentage=10
log.cleaner.dedupe.buffer.size=134217728
log.cleaner.delete.retention.ms=604800000
log.cleaner.enable=true
log.cleaner.min.cleanable.ratio=0.5
log.cleaner.threads=1
log.dirs=/kafka/data
log.retention.bytes=5368709120
log.retention.check.interval.ms=300000
log.retention.hours=72
log.retention.ms=259200000
log.roll.hours=168
log.segment.bytes=1073741824
message.max.bytes=3145728
min.insync.replicas=1
num.io.threads=8
num.partitions=1
num.replica.fetchers=6
offsets.topic.num.partitions=50
offsets.topic.replication.factor=3
port=9092
quota.consumer.default=52428800
quota.consumer.default=52428800
quota.producer.default=26214400
quota.producer.default=26214400

replica.fetch.max.bytes=4194304
replica.lag.max.messages=6000
replica.lag.time.max.ms=60000
unclean.leader.election.enable=false
zookeeper.session.timeout.ms=6000
zookeeper.connect=zookeeper01.prod.***.com:2181,zookeeper02.prod.***.com:2181,zookeeper03.prod.***.com:2181
security.inter.broker.protocol=PLAINTEXT

listeners=PLAINTEXT://kafka01.prod.***.com:9092,
broker.id.generation.enable=false
sasl.kerberos.service.name=kafka
listeners=PLAINTEXT://:9092
num.network.threads=8

通过检查卡夫卡来源(,),似乎唯一计入BytesRejectedPerSec
bytesRejectedRate
)的原因是消息大小超过了
config.maxMessageSize

注意:重新压缩和消息格式转换也可能影响消息大小,超出生产者发送的内容