使用spring Kafka中的注释为每个主题单独设置Kafka侦听器
我的应用程序正在收听多个卡夫卡主题。现在,我的侦听器如下所示,属性文件中的使用spring Kafka中的注释为每个主题单独设置Kafka侦听器,spring,spring-boot,kotlin,apache-kafka,spring-kafka,Spring,Spring Boot,Kotlin,Apache Kafka,Spring Kafka,我的应用程序正在收听多个卡夫卡主题。现在,我的侦听器如下所示,属性文件中的my.topics包含逗号分隔的卡夫卡主题列表 @KafkaListener(topics = ["#{'\${my.topics}'.split(',')}"], groupId = "my.group", containerFactory = "myKafkaFactory") fun genericMessageListener(myRequest: MyRequest, ack: Acknowledgment) {
my.topics
包含逗号分隔的卡夫卡主题列表
@KafkaListener(topics = ["#{'\${my.topics}'.split(',')}"], groupId = "my.group", containerFactory = "myKafkaFactory")
fun genericMessageListener(myRequest: MyRequest, ack: Acknowledgment) {
//do Something with myRequest
ack.acknowledge()
}
我的ConcurrentKafkaListenerContainerFactory
是
@Bean
fun myKafkaFactory(): ConcurrentKafkaListenerContainerFactory<String, MyRequest> {
val factory = ConcurrentKafkaListenerContainerFactory<String, MyRequest>()
factory.consumerFactory = DefaultKafkaConsumerFactory(configProps(), StringDeserializer(), MyRequestDeserializer())
factory.containerProperties.ackMode = ContainerProperties.AckMode.MANUAL
return factory
}
因此,factory.isOneConsumerPerTopic(true)
将确保为数组中的每个主题创建单独的使用者
我确实通过了。我正在寻找更“清洁”的解决方案 您可以向卡夫卡消费者配置中添加自定义的
分区赋值器
将容器并发性设置为(至少)主题数,并让您的指派者将每个主题的分区分配给特定的使用者
您可以从AbstractPartitionAssignor
开始
看
当收听多个主题时,默认的分区分布可能不是您所期望的。例如
@Bean
fun myKafkaFactory(): ConcurrentKafkaListenerContainerFactory<String, MyRequest> {
val factory = ConcurrentKafkaListenerContainerFactory<String, MyRequest>()
factory.consumerFactory = DefaultKafkaConsumerFactory(configProps(), StringDeserializer(), MyRequestDeserializer())
factory.containerProperties.ackMode = ContainerProperties.AckMode.MANUAL
factory.isOneConsumerPerTopic(true)
return factory
}
/**
* This interface is used to define custom partition assignment for use in
* {@link org.apache.kafka.clients.consumer.KafkaConsumer}. Members of the consumer group subscribe
* to the topics they are interested in and forward their subscriptions to a Kafka broker serving
* as the group coordinator. The coordinator selects one member to perform the group assignment and
* propagates the subscriptions of all members to it. Then {@link #assign(Cluster, Map)} is called
* to perform the assignment and the results are forwarded back to each respective members
*
* In some cases, it is useful to forward additional metadata to the assignor in order to make
* assignment decisions. For this, you can override {@link #subscription(Set)} and provide custom
* userData in the returned Subscription. For example, to have a rack-aware assignor, an implementation
* can use this user data to forward the rackId belonging to each member.
*/
public interface PartitionAssignor {