Apache kafka 卡夫卡经纪人领袖更迭无效

我有3个kafka代理，有3个分区：

broker.id 1001:10.18.0.73:9092领导者

broker.id 1002:10.18.0.73:9093

broker.id 1005:10.18.0.73:9094

动物园管理员设置为127.0.0.1:2181

推出时包括：

1001 -> .\kafka-server-start.bat ..\..\config\server.properties
1002 -> .\kafka-server-start.bat ..\..\config\server1.properties
1005 -> .\kafka-server-start.bat ..\..\config\server2.properties

这是server.properties

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9092
advertised.listeners=PLAINTEXT://10.18.0.73:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=10.18.0.73:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9092
advertised.host.name=10.18.0.73
port=9092

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9093
advertised.listeners=PLAINTEXT://10.18.0.73:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs4
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9093
advertised.host.name=10.18.0.73
port=9093

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9094
advertised.listeners=PLAINTEXT://10.18.0.73:9094
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs2
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9094
advertised.host.name=10.18.0.73
port=9094

这是server1.properties

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9092
advertised.listeners=PLAINTEXT://10.18.0.73:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=10.18.0.73:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9092
advertised.host.name=10.18.0.73
port=9092

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9093
advertised.listeners=PLAINTEXT://10.18.0.73:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs4
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9093
advertised.host.name=10.18.0.73
port=9093

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9094
advertised.listeners=PLAINTEXT://10.18.0.73:9094
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs2
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9094
advertised.host.name=10.18.0.73
port=9094

这是server2.properties

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9092
advertised.listeners=PLAINTEXT://10.18.0.73:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=10.18.0.73:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9092
advertised.host.name=10.18.0.73
port=9092

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9093
advertised.listeners=PLAINTEXT://10.18.0.73:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs4
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9093
advertised.host.name=10.18.0.73
port=9093

broker.id=-1
listeners=PLAINTEXT://10.18.0.73:9094
advertised.listeners=PLAINTEXT://10.18.0.73:9094
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs2
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
advertised.port=9094
advertised.host.name=10.18.0.73
port=9094

在文件夹C:\kafka_2.12-2.4.0\config中

全部运行

运营制作人

.\kafka-console-producer.bat --broker-list 10.18.0.73:9092,10.18.0.73:9093,10.18.0.73:9094 --topic clinicaleventmanager

运行消费者

.\kafka-console-consumer.bat --bootstrap-server 10.18.0.73:9092,10.18.0.73:9093,10.18.0.73:9094 --topic clinicaleventmanager

我启动了一个测试消息

接收ok

现在，我关闭了经纪人1001的领导

新领导人是1002

在《消费者》杂志上，这条信息出现了一秒钟，我想这是选举新领导人所需要的时间

[2020-01-16 15:33:35,802] WARN [Consumer clientId=consumer-console-consumer-56669-1, groupId=console-consumer-56669] Connection to node 2147482646 (/10.18.0.73:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

如果我尝试发送另一条消息，则用户不会读取该消息

新的引线1002似乎没有发送消息。 为什么?

如果我运行1001 broker.id，则可以全部运行。

---

谢谢

首先，卡夫卡从不发送推送消息，消费者会要求发送推送消息

其次，除了侦听器、端口和log dir之外，您似乎什么都没有更改

您没有显式地创建任何主题，因此最终会得到一个分区和一个副本的默认值。针对您的主题和内部消费者补偿主题

如果任何复制副本从您停止的代理脱机，则无论哪个代理是控制器，其他进程都无法读取或写入该复制副本

---

因此，请将偏移量和事务复制系数更改为3，然后重试

谢谢，但我知道kafka不会自动向消费者发送消息。我已经修改了offsets.topic.replication.factor=2 default.replication.factor=2的属性，它成功了。感谢