Java 多个消费者如何在spring boot Kafka中收听多个主题?

Java 多个消费者如何在spring boot Kafka中收听多个主题?,java,spring-boot,apache-kafka,Java,Spring Boot,Apache Kafka,当有多个消费者时,我无法听卡夫卡主题(我的案例2主题)。 在下面的示例中,我有2个消费者工厂,这将是2个不同的JSON消息(一个是用户类型,另一个是事件类型)。这两条消息都发布到不同的主题。在这里,当我试图从topic1访问事件消息时,我无法访问,但我可以访问用户主题消息 例: @Configuration @EnableKafka public class KafkaConsumerConfiguration { @Autowired private Environment env

当有多个消费者时,我无法听卡夫卡主题(我的案例2主题)。 在下面的示例中,我有2个消费者工厂,这将是2个不同的JSON消息(一个是用户类型,另一个是事件类型)。这两条消息都发布到不同的主题。在这里,当我试图从topic1访问事件消息时,我无法访问,但我可以访问用户主题消息

例:

@Configuration
@EnableKafka
public class KafkaConsumerConfiguration {      
@Autowired
private Environment environment;

@Bean
public ConsumerFactory<String,User> consumerFactory() {
    Map<String, Object> config = new HashMap<>();
    config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, environment.getProperty("bootstrap.servers"));
    config.put(ConsumerConfig.GROUP_ID_CONFIG, environment.getProperty("user.consumer.group"));
    config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,JsonDeserializer.class);

    return new DefaultKafkaConsumerFactory<>(config, new StringDeserializer(),
            new JsonDeserializer<>(User.class));

}
@Bean
public ConcurrentKafkaListenerContainerFactory<String, User> kafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, User> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory());
    return factory;
}

@Bean
public ConsumerFactory<String , Event> consumerFactoryEvent(){
    Map<String, Object> config = new HashMap<>();
    config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, environment.getProperty("bootstrap.servers"));
    config.put(ConsumerConfig.GROUP_ID_CONFIG, environment.getProperty("event.consumer.group"));
    config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,JsonDeserializer.class);

    return new DefaultKafkaConsumerFactory<>(config, new StringDeserializer(),
            new JsonDeserializer<>(Event.class));
}

@Bean
public ConcurrentKafkaListenerContainerFactory<String, Event> kafkaListenerContainerFactoryEvent() {
    ConcurrentKafkaListenerContainerFactory<String, Event> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactoryEvent());
    return factory;
}
}
我需要先听事件主题,对消息进行一些处理,然后将其发送给用户主题,我有另一种方法,可以听用户主题,并对该消息进行处理。。 我尝试将不同的选项传递给@KafkaListener,例如

@KafkaListener(topics="${event.topic}",containerFactory="kafkaListenerContainerFactoryEvent")

但是它不起作用。。我不确定出了什么问题。。任何建议都是有帮助的

如果您没有在bean中指定名称,那么方法名称将是bean名称,在
@KafkaListener

@KafkaListener(topics="${event.topic}",containerFactory="kafkaListenerContainerFactoryEvent", groupId="")

@KafkaListener(topics="${event.topic}",containerFactory="kafkaListenerContainerFactory", groupId="")

@Bean
中指定名称,并将该名称添加到
@kafkaListener

@Bean(name="kafkaListenerContainerFactoryEvent")
public ConcurrentKafkaListenerContainerFactory<String, Event> kafkaListenerContainerFactoryEvent() {
ConcurrentKafkaListenerContainerFactory<String, Event> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactoryEvent());
return factory;
}
@Bean(name=“kafkaListenerContainerFactoryEvent”)
public ConcurrentKafkaListenerContainerFactory kafkaListenerContainerFactoryEvent(){
ConcurrentKafkListenerContainerFactory=新ConcurrentKafkListenerContainerFactory();
setConsumerFactory(consumerFactoryEvent());
返回工厂;
}
在任何文档中都不容易找到它

这里,我以使用来自的消息为例

topic=topic1,bootstrapserver=url1(JSON序列化程序和反序列化程序)

topic=topic2和bootstrapserver=url2(Avro序列化程序和反序列化程序)

第1步:-


显然,现在回答这个问题太晚了。但它可能会帮助其他人


您不需要创建多个ConsumerFactorybean。您可以在配置中不通知类(
用户
事件
),即
新建JsonDeserializer(Event.class)
并添加受信任的包

@Bean
公共消费者工厂消费者工厂(){
Map config=newhashmap();
config.put(ConsumerConfig.BOOTSTRAP\u SERVERS\u config,environment.getProperty(“BOOTSTRAP.SERVERS”);
config.put(ConsumerConfig.GROUP\u ID\u config,environment.getProperty(“user.consumer.GROUP”);
config.put(ConsumerConfig.KEY\u反序列化程序\u CLASS\u config,StringDeserializer.CLASS);
put(ConsumerConfig.VALUE\u反序列化程序\u类\u配置,JsonDeserializer.CLASS);
//TODO:删除“*”并添加特定的包名称

config.put(jsondesializer.TRUSTED_PACKAGES,“*”;//尝试为两个侦听器定义两个不同的bean名称,并将这些名称添加到“@kafkaListener”annotationPerfect中,这正是我想要的
@Bean(name="kafkaListenerContainerFactoryEvent")
public ConcurrentKafkaListenerContainerFactory<String, Event> kafkaListenerContainerFactoryEvent() {
ConcurrentKafkaListenerContainerFactory<String, Event> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactoryEvent());
return factory;
}
@Bean
public ConsumerFactory<String, String> consumerFactory1() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
            "localhost1:9092"); //This is dummy
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

    return new DefaultKafkaConsumerFactory<>(props);
}

@Bean
public ConsumerFactory consumerFactory2() {
    Map props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
            "localhost2:9092"); //This is dummy
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class);
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    props.put("schema.registry.url", "https://abc.schemaregistery.example.com"); //Again this is dummy or can be avro serilaised class
    return new DefaultKafkaConsumerFactory<>(props);
}


  @Bean(name = "kafkaListenerContainerFactory1")
public ConcurrentKafkaListenerContainerFactory
kafkaListenerContainerFactory1() {
    ConcurrentKafkaListenerContainerFactory<String, String> factory
            = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory1());
    return factory;
}

 @Bean(name = "kafkaListenerContainerFactory2")
public ConcurrentKafkaListenerContainerFactory
kafkaListenerContainerFactory2() {
    ConcurrentKafkaListenerContainerFactory<String, String> factory
            = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory2());
    return factory;
}
 @KafkaListener(
            topics = "topic1",
            containerFactory = "kafkaListenerContainerFactory1" ,
            groupId = "com.groupid1")
    public void receive(ConsumerRecord consumerRecord) throws InterruptedException {


        LOGGER.info("consuming from topic1 {}" , consumerRecord.value());
        Thread.sleep(1000000); //For testing

    }

 @KafkaListener(
            topics = "topic2",
            containerFactory = "kafkaListenerContainerFactory2" ,
            groupId = "com.groupid2")
    public void receive(ConsumerRecord consumerRecord) throws InterruptedException {


        LOGGER.info("consuming from topic2 {}" , consumerRecord.value());
        Thread.sleep(1000000); //For testing

    }