Java Spring Boot中与Apache Kafka的Json消息通信_Java_Spring_Spring Boot_Apache Kafka_Spring Kafka

Java Spring Boot中与Apache Kafka的Json消息通信

java spring spring-boot apache-kafka

Java Spring Boot中与Apache Kafka的Json消息通信,java,spring,spring-boot,apache-kafka,spring-kafka,Java,Spring,Spring Boot,Apache Kafka,Spring Kafka,我已经实现了两个不同的应用程序，一个用于生产者，另一个用于消费者，并且在apache kafka的支持下传递了消息。当我发布一条字符串消息时，通信正常完成，但当我传递Json消息时，出现了以下错误错误 java.lang.IllegalStateException: This error handler cannot process 'SerializationException's directly; please consider configuring an 'ErrorHandling

我已经实现了两个不同的应用程序，一个用于生产者，另一个用于消费者，并且在apache kafka的支持下传递了消息。当我发布一条字符串消息时，通信正常完成，但当我传递Json消息时，出现了以下错误

错误

java.lang.IllegalStateException: This error handler cannot process 'SerializationException's directly; please consider configuring an 'ErrorHandlingDeserializer' in the value and/or key deserializer

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition Kafka_Example_Json-0 at offset 0. If needed, please seek past the record to continue consumption.

Caused by: java.lang.IllegalArgumentException: The class 'com.benz.kafka.api.model.User' is not in the trusted packages: [java.util, java.lang, com.benz.kafka.consumer.api.model, com.benz.kafka.consumer.api.model.*]. If you believe this class is safe to deserialize, please provide its name. If the serialization is only done by a trusted source, you can also enable trust all (*).

消费者配置类

@Configuration
@EnableKafka
public class KafkaConfig {

private ConsumerFactory<String, User> userConsumerFactory()
    {
        Map<String,Object> config=new ConcurrentHashMap<>();

        config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
        config.put(ConsumerConfig.GROUP_ID_CONFIG,"group_json");
        config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        config.put(ErrorHandlingDeserializer.KEY_DESERIALIZER_CLASS, JsonDeserializer.class);
        config.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS,JsonDeserializer.class.getClass());
        config.put(JsonDeserializer.TRUSTED_PACKAGES,"*");

          return new DefaultKafkaConsumerFactory<>(config,new StringDeserializer(),new JsonDeserializer<>(User.class));
    }

 @Bean
    public ConcurrentKafkaListenerContainerFactory<String,User> userKafkaListenerContainerFactory()
    {
        ConcurrentKafkaListenerContainerFactory<String,User> factory
                =new ConcurrentKafkaListenerContainerFactory<>();

        factory.setConsumerFactory(userConsumerFactory());

        return factory;

    }
}

@Configuration
public class KafkaConfig {

    private ProducerFactory<String,User> producerFactory()
    {
        Map<String,Object> config=new ConcurrentHashMap<>();

          config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
          config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
          config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);

          return new DefaultKafkaProducerFactory<>(config);
    }

    @Bean
    public KafkaTemplate<String,User> kafkaTemplate()
    {
        return new KafkaTemplate<>(producerFactory());
    }


}

产品配置类

@Configuration
@EnableKafka
public class KafkaConfig {

private ConsumerFactory<String, User> userConsumerFactory()
    {
        Map<String,Object> config=new ConcurrentHashMap<>();

        config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
        config.put(ConsumerConfig.GROUP_ID_CONFIG,"group_json");
        config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        config.put(ErrorHandlingDeserializer.KEY_DESERIALIZER_CLASS, JsonDeserializer.class);
        config.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS,JsonDeserializer.class.getClass());
        config.put(JsonDeserializer.TRUSTED_PACKAGES,"*");

          return new DefaultKafkaConsumerFactory<>(config,new StringDeserializer(),new JsonDeserializer<>(User.class));
    }

 @Bean
    public ConcurrentKafkaListenerContainerFactory<String,User> userKafkaListenerContainerFactory()
    {
        ConcurrentKafkaListenerContainerFactory<String,User> factory
                =new ConcurrentKafkaListenerContainerFactory<>();

        factory.setConsumerFactory(userConsumerFactory());

        return factory;

    }
}

@Configuration
public class KafkaConfig {

    private ProducerFactory<String,User> producerFactory()
    {
        Map<String,Object> config=new ConcurrentHashMap<>();

          config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
          config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
          config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);

          return new DefaultKafkaProducerFactory<>(config);
    }

    @Bean
    public KafkaTemplate<String,User> kafkaTemplate()
    {
        return new KafkaTemplate<>(producerFactory());
    }


}

您需要使用@Bean对其进行注释并添加以下配置：

    @Bean
    private ProducerFactory<String,User> producerFactory()
{
    Map<String,Object> config=new ConcurrentHashMap<>();

      config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
      config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
      config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);

      return new DefaultKafkaProducerFactory<>(config);
}

@Bean
    public Map<String, Object> producerConfigs() {
        Map<String, Object> props = new HashMap<>();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
        return props;
    }

    @Bean
    public ProducerFactory<String, AccountEvent> producerFactory() {
        return new DefaultKafkaProducerFactory<>(producerConfigs());
    }

    @Bean
    public KafkaTemplate<String, AccountEvent> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }

@Bean
私人生产厂生产厂（）
{
映射配置=新的ConcurrentHashMap（）；
config.put（ProducerConfig.BOOTSTRAP\u SERVERS\u config，“127.0.0.1:9092”）；
config.put（ProducerConfig.KEY\u SERIALIZER\u CLASS\u config，StringSerializer.CLASS）；
config.put（ProducerConfig.VALUE\u SERIALIZER\u CLASS\u config，JsonSerializer.CLASS）；
返回新的DefaultKafkaProducerFactory（配置）；
}
@豆子
公共地图产品配置（）{
Map props=newhashmap（）；
put（ProducerConfig.BOOTSTRAP\u server\u CONFIG，bootstrapserver）；
put（ProducerConfig.KEY\u SERIALIZER\u CLASS\u CONFIG，StringSerializer.CLASS）；
put（ProducerConfig.VALUE\u SERIALIZER\u CLASS\u CONFIG，JsonSerializer.CLASS）；
返回道具；
}
@豆子
公共生产工厂生产工厂（）{
返回新的DefaultKafkaProducerFactory（producerConfigs（））；
}
@豆子
公共卡夫卡模板卡夫卡模板（）{
返回新的卡夫卡模板（producerFactory（））；
}

如果您想使用带有kafka基础设施的项目，我在github中有它。这个

config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);

应该是

config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);

但是，不清楚为什么不应用您的可信包配置；建议您在

JsonDeserializer.configure（）方法中设置断点
编辑
哦
我已经找到了发生此错误的原因。您可以在错误下看到，零件模型类是在两个不同的包中创建的。在Producer中，模型类是在com.benz.kafka.api.model
包下创建的，在Consumer部分，模型是在com.benz.kafka.Consumer.api.model
包下创建的。这就是根本原因，我将com.benz.kafka.consumer.api.model
更改为com.benz.kafka.api.model
，然后它就工作了。
无需为这两种方法创建bean实例。顺便说一句，我的答案没有被编辑；你。正在重写构造函数中的反序列化程序属性。我更改了它，但不起作用。当我使用kafka console producer作为生产者时，消费者应用程序会正确地使用它，但当我使用producer作为spring boot应用程序并发送时，则会发生此错误，这毫无意义，
return new DefaultKafkaConsumerFactory<>(config,new StringDeserializer(),new JsonDeserializer<>(User.class));

return new DefaultKafkaConsumerFactory<>(config);