Apache kafka 应用程序处于活动状态，但一段时间后停止使用消息

spring kafka消费者在一段时间后停止消费消息。每次停车都会发生，但不会在同一时间发生。当应用程序不再消费时，在日志的末尾，我总是看到消费者发送离开组信号的语句。如果我没有看到任何错误或异常，为什么消费者要离开集团

org.springframework.boot:spring-boot-starter-parent:2.0.4.RELEASE
spring-kafka:2.1.8.RELEASE
org.apache.kafka:kafka-clients:1.0.2

我已将日志设置为 logging.level.org.apache.kafka=DEBUG logging.level.org.springframework.kafka=INFO

其他设置

spring.kafka.listener.concurrency=5
spring.kafka.listener.type=single
spring.kafka.listener.ack-mode=record
spring.kafka.listener.poll-timeout=10000
spring.kafka.consumer.heartbeat-interval=5000
spring.kafka.consumer.max-poll-records=50
spring.kafka.consumer.fetch-max-wait=10000
spring.kafka.consumer.enable-auto-commit=false
spring.kafka.consumer.properties.security.protocol=SSL
spring.kafka.consumer.retry.maxAttempts=3
spring.kafka.consumer.retry.backoffperiod.millisecs=2000

集装箱工厂设置

@Bean
公共ConcurrentKafkaListenerContainerFactory记录KafkaListenerContainerFactory（RetryTemplate RetryTemplate）{
ConcurrentKafkListenerContainerFactory=新ConcurrentKafkListenerContainerFactory（）；
setConsumerFactory（consumerFactory）；
factory.setConcurrency（listenerCount）；
factory.getContainerProperties（）.setAckMode（AbstractMessageListenerContainer.AckMode.RECORD）；
factory.getContainerProperties（）.setPollTimeout（pollTimeoutMillis）；
factory.getContainerProperties（）.setErrorHandler（新的seekToCurInterrorHandler（））；
factory.getContainerProperties（）.setAckoneError（false）；
factory.setRetryTemplate（retryTemplate）；
factory.setStatefulRetry（true）；
factory.getContainerProperties（）.setIdleEventInterval（60000L）；
返回工厂；
}

侦听器配置

@Component
public class RecordsEventListener implements ConsumerSeekAware {

    private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(RecordsEventListener.class);



    @Value("${mode.replay:false}")
    public void setModeReplay(boolean enabled) {

        this.isReplay = enabled;
    }

    @KafkaListener(topics = "${event.topic}", containerFactory = "RecordsKafkaListenerContainerFactory")
    public void handleEvent(@Payload String payload) throws RecordsEventListenerException {

        try {
           //business logic
        } catch (Exception e) {

            LOG.error("Process error for event: {}",payload,e);

            if(isRetryableException(e)) {
                LOG.warn("Retryable exception detected. Going to retry.");
                throw new RecordsEventListenerException(e);
            }else{
                LOG.warn("Dropping event because non retryable exception");
            }
        }

    }

    private Boolean isRetryableException(Exception e) {

        return binaryExceptionClassifier.classify(e);
    }





    @Override
    public void registerSeekCallback(ConsumerSeekCallback callback) {
        //do nothing
    }

    @Override
    public void onPartitionsAssigned(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {

        //do this only once per start of app
        if (isReplay && !partitonSeekToBeginningDone) {
            assignments.forEach((t, p) -> callback.seekToBeginning(t.topic(), t.partition()));
            partitonSeekToBeginningDone = true;
        }

    }

    @Override
    public void onIdleContainer(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
        //do nothing
        LOG.info("Container is IDLE; no messages to pull.");
        assignments.forEach((t,p)->LOG.info("Topic:{}, Partition:{}, Offset:{}",t.topic(),t.partition(),p));
    }

    boolean isPartitionSeekToBeginningDone() {

        return partitonSeekToBeginningDone;
    }

    void setPartitonSeekToBeginningDone(boolean partitonSeekToBeginningDone) {

        this.partitonSeekToBeginningDone = partitonSeekToBeginningDone;
    }
}

---

感谢所有回复的人。事实证明，确实是代理在会话超时时丢弃了使用者。broker a非常旧的版本（0.10.0.1）不支持我们可以使用的spring kafka版本中概述的更新功能。

---

由于我们无法指定对代理的升级或对会话超时的更改，我们只是修改了处理逻辑，以便在会话超时下完成任务。

我添加了[apache kafka]标记，因为我认为这与spring无关<代码>>使用较旧的服务器API v0-您的代理版本是什么？我相信，如果（1）消费者已关闭，或者（2）由于

max.poll.interval.ms

的流逝，消费者在重新平衡后未及时重新加入，您将收到此消息。日志是否显示了其中任何一项？@GaryRussell，代理由另一个团队处理。他们说是0.10.01。@frankgreco，我添加了更多的日志作为要点的链接。如果您在日志中看到行为2，请告诉我。还要澄清的是，max.poll.interval.ms与spring.kafka.consumer.heartbeat-interval相同，客户端在与代理交谈时设置它？不，它是kafka消费者属性-它是

poll（）

s之间允许的最长时间。您的侦听器必须在此时间内退出（默认为3000000毫秒）。请参阅，它已添加到0.10.1.0代理中。您可以使用

spring.kafka.consumer.properties.max.poll.interval.ms=…

进行设置。我看不出要点中有什么明显的东西；让他们查看相同时间段的服务器日志。

2019-05-02 18:31:05.770 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Sending Heartbeat request to coordinator x.x.x.com:9093 (id: 2147482638 rack: null)
2019-05-02 18:31:05.770 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-1, groupId=app] Using older server API v0 to send HEARTBEAT {group_id=app,generation_id=6,member_id=consumer-1-98d28e69-b0b9-4c2b-82cd-731e53b74b87} with correlation id 5347 to node 2147482638
2019-05-02 18:31:05.872 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Received successful Heartbeat response
2019-05-02 18:31:10.856 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Sending Heartbeat request to coordinator x.x.x.com:9093 (id: 2147482638 rack: null)
2019-05-02 18:31:10.857 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-1, groupId=app] Using older server API v0 to send HEARTBEAT {group_id=app,generation_id=6,member_id=consumer-1-98d28e69-b0b9-4c2b-82cd-731e53b74b87} with correlation id 5348 to node 2147482638
2019-05-02 18:31:10.958 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Received successful Heartbeat response
2019-05-02 18:31:11.767 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Sending LeaveGroup request to coordinator x.x.x.com:9093 (id: 2147482638 rack: null)
2019-05-02 18:31:11.767 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-1, groupId=app] Using older server API v0 to send LEAVE_GROUP {group_id=app,member_id=consumer-1-98d28e69-b0b9-4c2b-82cd-731e53b74b87} with correlation id 5349 to node 2147482638
2019-05-02 18:31:11.768 DEBUG 9548 --- [kafka-coordinator-heartbeat-thread | app] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=app] Disabling heartbeat thread