Apache kafka 如何将具有leader-1的分区移动到另一个代理?
假设所有具有分区的前导和副本的节点都崩溃,并且节点不可恢复 如何选举另一位卡夫卡经纪人作为该分区的领导人?(我对数据丢失感到满意) 我尝试了“kafka preferred replica election.sh”和“kafka reassign partitions.sh”,但没有成功 设置:Apache kafka 如何将具有leader-1的分区移动到另一个代理?,apache-kafka,Apache Kafka,假设所有具有分区的前导和副本的节点都崩溃,并且节点不可恢复 如何选举另一位卡夫卡经纪人作为该分区的领导人?(我对数据丢失感到满意) 我尝试了“kafka preferred replica election.sh”和“kafka reassign partitions.sh”,但没有成功 设置: bin/kafka-reassign-partitions.sh --zookeeper myvm:2181 --reassignment-json-file manual-assignment.jso
bin/kafka-reassign-partitions.sh --zookeeper myvm:2181 --reassignment-json-file manual-assignment.json --execute
Partitions reassignment failed due to The proposed assignment contains non-existent partitions: List([test,10])
kafka.common.AdminCommandFailedException: The proposed assignment contains non-existent partitions: List([test,10])
at kafka.admin.ReassignPartitionsCommand$.parseAndValidate(ReassignPartitionsCommand.scala:201)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:153)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:149)
at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:46)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
cat manual-assignment.json
{
"partitions":
[
{"topic": "test", "partition": 10, "replicas": [10]}
],
"version":1
}
- 代理id为10和11的两节点kafka群集
- 主题“使用副本因子1和分区1进行测试”。代理11是主题“测试”分区0的负责人
- 经纪人11下跌
- 卡夫卡版本-卡夫卡2.11-0.10.1.0
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: -1 Replicas: 11 Isr:
kafka-reassign-partitions.sh输出:
bin/kafka-reassign-partitions.sh --zookeeper myvm:2181 --reassignment-json-file manual-assignment.json --execute
Partitions reassignment failed due to The proposed assignment contains non-existent partitions: List([test,10])
kafka.common.AdminCommandFailedException: The proposed assignment contains non-existent partitions: List([test,10])
at kafka.admin.ReassignPartitionsCommand$.parseAndValidate(ReassignPartitionsCommand.scala:201)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:153)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:149)
at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:46)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
cat manual-assignment.json
{
"partitions":
[
{"topic": "test", "partition": 10, "replicas": [10]}
],
"version":1
}
卡夫卡更喜欢复制品选举。sh
bin/kafka-preferred-replica-election.sh --zookeeper myvm:2181 --path-to-json-file topicPartitionList.json
[2016-11-08 03:27:01,993] ERROR Skipping preferred replica leader election for partition [test,10] since it doesn't exist (kafka.admin.PreferredReplicaLeaderElectionCommand)
Successfully started preferred replica election for partitions Set([test,10])
cat topicPartitionList.json
{
"partitions":
[
{"topic": "test", "partition": 10}
]
}
卡夫卡仅从ISR和指定副本中选举领导人(启用不干净领导人选举时)。对于您的情况,当复制因子设置为1时,在leader broker崩溃后,ISR中没有任何代理。没有候选人可以当选为新的领导人,因此这是设计。因此,让主题重新开始的唯一方法是删除主题并重新开始?