Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka 卡夫卡反应器-如何禁用自动启动的卡夫卡消费者?_Apache Kafka_Spring Webflux_Project Reactor_Reactor_Reactor Kafka - Fatal编程技术网

Apache kafka 卡夫卡反应器-如何禁用自动启动的卡夫卡消费者?

Apache kafka 卡夫卡反应器-如何禁用自动启动的卡夫卡消费者?,apache-kafka,spring-webflux,project-reactor,reactor,reactor-kafka,Apache Kafka,Spring Webflux,Project Reactor,Reactor,Reactor Kafka,下面是我的卡夫卡消费者 @Bean("kafkaConfluentInboundReceiver") @ConditionalOnProperty(value = "com.demo.kafka.core.inbound.confluent.topic-name", matchIfMissing = false) public KafkaReceiver<String, Object> kafkaInboundReceiver(

下面是我的卡夫卡消费者

@Bean("kafkaConfluentInboundReceiver")
@ConditionalOnProperty(value = "com.demo.kafka.core.inbound.confluent.topic-name",
        matchIfMissing = false)
public KafkaReceiver<String, Object> kafkaInboundReceiver() {
    ReceiverOptions<String, Object> receiverOptions = ReceiverOptions.create(inboundConsumerConfigs());
    receiverOptions.schedulerSupplier(() -> Schedulers
            .fromExecutorService(applicationContext.getBean("inboundKafkaExecutorService", ExecutorService.class)));
    receiverOptions.maxCommitAttempts(kafkaProperties.getKafka().getCore().getMaxCommitAttempts());
    return KafkaReceiver.create(receiverOptions.addAssignListener(Collection::iterator)
            .subscription(Collections.singleton(
                    kafkaProperties.getKafka()
                            .getCore().getInbound().getConfluent()
                            .getTopicName()))
            .commitInterval(Duration.ZERO).commitBatchSize(0));
}
然而,我不确定如何在卡夫卡反应堆中实现(控制自动启动/停止行为)。我想要下面这样的东西

引入属性来处理自动启动/停止行为

@Value("${consumer.autostart:true}")
private boolean start;
使用上面的属性,我应该能够在卡夫卡反应器中设置卡夫卡自动启动标志,如下所示

factory.setAutoStartup(start);
return KafkaReceiver.create(receiverOptions.addAssignListener(Collection::iterator)
        .subscription(Collections.singleton(
                kafkaProperties.getKafka()
                        .getCore().getInbound().getConfluent()
                        .getTopicName()))
        .commitInterval(Duration.ZERO).commitBatchSize(0)).setAutoStart(start);
注:.setAutoStart(启动)

这在卡夫卡反应堆中可行吗?如果可行,我该怎么做

更新:

protected void inboundEventHubListener(String topicName, List<String> allowedValues) {
    Scheduler scheduler = Schedulers.fromExecutorService(kafkaExecutorService);
    kafkaEventHubInboundReceiver
            .receive()
            .publishOn(scheduler)
            .groupBy(receiverRecord -> {
                try {
                    return receiverRecord.receiverOffset().topicPartition();
                } catch (Throwable throwable) {
                    log.error("exception in groupby", throwable);
                    return Flux.empty();
                }
            }).flatMap(partitionFlux -> partitionFlux.publishOn(scheduler)
            .map(record -> {
                processMessage(record, topicName, allowedValues).block(
                        Duration.ofSeconds(60L));//This subscribe is to trigger processing of a message
                return record;
            }).concatMap(message -> {
                log.info("Received message after processing offset: {} partition: {} ",
                         message.offset(), message.partition());
                return message.receiverOffset()
                        .commit()
                        .onErrorContinue((t, o) -> log.error(
                                String.format("exception raised while commit offset %s", o), t)
                        );
            })).onErrorContinue((t, o) -> {
        try {
            if (null != o) {
                ReceiverRecord<String, Object> record = (ReceiverRecord<String, Object>) o;
                ReceiverOffset offset = record.receiverOffset();
                log.debug("failed to process message: {} partition: {} and message: {} ",
                          offset.offset(), record.partition(), record.value());
            }
            log.error(String.format("exception raised while processing message %s", o), t);
        } catch (Throwable inner) {
            log.error("encountered error in onErrorContinue", inner);
        }
    }).subscribeOn(scheduler).subscribe();

对于
反应堆卡夫卡
没有“自动启动”的概念;你完全在控制之中

在订阅从
receiver.receive()
返回的
Flux
之前,使用者不会“启动”


只需延迟
flux.subscribe()
,直到您准备好使用数据。

谢谢您,Gary Russell。我已经用代码片段更新了问题,您能告诉我这是否可行吗?这是正确的,但对象将是
flux
kafkaEventHubInboundReceiverObj = kafkaEventHubInboundReceiver.....subscribeOn(scheduler);
if(consumer.autostart) {
kafkaEventHubInboundReceiverObj.subscribe();
}