Fatal编程技术网
  • apache-kafka/
  • Apache kafka 卡夫卡流在EOS模式下关闭，无任何异常

Apache kafka 卡夫卡流在EOS模式下关闭，无任何异常

apache-kafka

Apache kafka 卡夫卡流在EOS模式下关闭，无任何异常,apache-kafka,apache-kafka-streams,Apache Kafka,Apache Kafka Streams,我有一个kafka streams应用程序，它会无故关闭（至少我在日志中找不到它），并且总是在重新启动后恢复。完整的日志在底部。这里有几句话引起了我的注意 2020-11-03 23:04:32:341 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO c.a.t.u.EventSequencerStateRestoreListener:? - event-seq-state-stor

我有一个kafka streams应用程序，它会无故关闭（至少我在日志中找不到它），并且总是在重新启动后恢复。完整的日志在底部。这里有几句话引起了我的注意

2020-11-03 23:04:32:341 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  c.a.t.u.EventSequencerStateRestoreListener:? - event-seq-state-store done restoring for topic : demo-es-1-null-event-seq-state-store-changelog and partition 0. Total restored : 44780 
2020-11-03 23:04:32:343 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Transitioning stream task 0_0 to running
2020-11-03 23:04:32:434 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  c.a.t.e.EventSequenceProcessor:? - EventSequencer Processor getting initialized with 
2020-11-03 23:04:32:438 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING
2020-11-03 23:04:38:135 +0000 [kafka-streams-close-thread] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Informed to shut down
2020-11-03 23:04:38:135 +0000 [kafka-streams-close-thread] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from RUNNING to PENDING_SHUTDOWN
2020-11-03 23:04:38:135 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State already transits to PENDING_SHUTDOWN, skipping the run once call after poll request
2020-11-03 23:04:38:135 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Ignoring request to transit from PENDING_SHUTDOWN to PENDING_SHUTDOWN: only DEAD state is a valid next state
2020-11-03 23:04:38:136 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Shutting down
2020-11-03 23:04:38:136 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.TaskManager:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Shutting down all active tasks [0_0], standby tasks [], suspended tasks [], and suspended standby tasks []
拓扑是输入主题->状态存储->输出主题。由于我正在使用内存状态存储，应用程序将恢复changelog主题。流线程实际上进入运行状态，但是kafka streams close线程正在接收一个关闭命令，该命令由->表示为“通知关闭”。重新启动后，此错误消失。完整的日志如下。我有一个uncaughtExceptionHandler，我正在打印异常，但在日志中也没有看到任何异常。 此外，该流正在EOS模式下运行。有人知道为什么会这样吗

代理版本：2.2.1

Streams库版本：2.3.1

有时它在关闭时也会抛出这个KafkaException“无法向事务发送偏移量，因为生产者不在活动事务中”

日志：

2020-10-27 23:52:53:690 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] ERROR o.a.k.s.p.internals.StreamTask:? - task [0_0] Could not close task due to the following error:
org.apache.kafka.common.KafkaException: Cannot send offsets to transaction either because the producer is not in an active transaction
    at org.apache.kafka.clients.producer.internals.TransactionManager.sendOffsetsToTransaction(TransactionManager.java:329)
    at org.apache.kafka.clients.producer.KafkaProducer.sendOffsetsToTransaction(KafkaProducer.java:673)
    at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:486)
    at org.apache.kafka.streams.processor.internals.StreamTask.suspend(StreamTask.java:596)
    at org.apache.kafka.streams.processor.internals.StreamTask.close(StreamTask.java:724)
    at org.apache.kafka.streams.processor.internals.AssignedTasks.close(AssignedTasks.java:341)
    at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(TaskManager.java:267)
    at org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(StreamThread.java:1230)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:800)
2020-10-27 23:52:53:691 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Closing its state manager and all the registered state stores
2020-10-27 23:52:53:691 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Closing storage engine event-seq-state-store
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] ERROR o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] Failed while closing StreamTask 0_0 due to the following error:
org.apache.kafka.common.KafkaException: Cannot send offsets to transaction either because the producer is not in an active transaction
    at org.apache.kafka.clients.producer.internals.TransactionManager.sendOffsetsToTransaction(TransactionManager.java:329)
    at org.apache.kafka.clients.producer.KafkaProducer.sendOffsetsToTransaction(KafkaProducer.java:673)
    at org.apache.kafka.streams.processor.internals.StreamTask.commit(StreamTask.java:486)
    at org.apache.kafka.streams.processor.internals.StreamTask.suspend(StreamTask.java:596)
    at org.apache.kafka.streams.processor.internals.StreamTask.close(StreamTask.java:724)
    at org.apache.kafka.streams.processor.internals.AssignedTasks.close(AssignedTasks.java:341)
    at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(TaskManager.java:267)
    at org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(StreamThread.java:1230)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:800)
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] INFO  o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] Try to close StreamTask 0_0 unclean.
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamTask:? - task [0_0] Closing
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] DEBUG o.a.k.s.p.i.RecordCollectorImpl:? - task [0_0] Closing producer
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Closing its state manager and all the registered state stores
2020-10-27 23:52:53:693 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] INFO  o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Skipping to close non-initialized store event-seq-state-store
2020-10-27 23:52:53:696 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD
2020-10-27 23:52:53:696 +0000 [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-2fe427fa-a436-4e21-a07f-476ab13cdbbf-StreamThread-1] Shutdown complete
完整日志：

Listening for transport dt_socket at address: [REDACTED]
2020-11-03 23:04:19:243 +0000 [main] INFO  c.a.t.config.EventSequencerConfig:? - Topology initialized: org.apache.kafka.streams.Topology@12921ee1
2020-11-03 23:04:21:736 +0000 [main] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Creating restore consumer client
2020-11-03 23:04:23:135 +0000 [main] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Creating consumer client
2020-11-03 23:04:23:237 +0000 [main] DEBUG o.a.k.s.p.i.InternalTopicManager:? - stream-thread [main] Configs:

2020-11-03 23:04:23:338 +0000 [main] WARN  o.a.k.c.consumer.ConsumerConfig:? - The configuration 'admin.retries' was supplied but isn't a known config.
2020-11-03 23:04:23:338 +0000 [main] WARN  o.a.k.c.consumer.ConsumerConfig:? - The configuration 'admin.retry.backoff.ms' was supplied but isn't a known config.
2020-11-03 23:04:23:434 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Starting
2020-11-03 23:04:23:434 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from CREATED to STARTING
2020-11-03 23:04:23:435 +0000 [main] INFO  c.a.t.c.IngestionTopicConsumer:? - StreamThread Metadata : ThreadMetadata{threadName=demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1, threadState=STARTING, activeTasks=[], standbyTasks=[], consumerClientId=demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer, restoreConsumerClientId=demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-restore-consumer, producerClientIds=[], adminClientId=demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-admin} 
2020-11-03 23:04:24:833 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] at state STARTING: partitions [] revoked at the beginning of consumer rebalance.
    current assigned active tasks: []
    current assigned standby tasks: []

2020-11-03 23:04:24:833 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from STARTING to PARTITIONS_REVOKED
2020-11-03 23:04:24:833 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.TaskManager:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Suspending all active tasks [] and standby tasks []
2020-11-03 23:04:24:833 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] partition revocation took 0 ms.
    suspended active tasks: []
    suspended standby tasks: []
2020-11-03 23:04:24:835 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.InternalTopologyBuilder:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] found [input-event-stream] topics possibly matching regex
2020-11-03 23:04:24:835 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.InternalTopologyBuilder:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] updating builder with SubscriptionUpdates{updatedTopicSubscriptions=[input-event-stream]} topic(s) with possible matching regex subscription(s)
Event Sequencer Server started, listening on 2301
2020-11-03 23:04:28:340 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Current minimum supported version remains at 4, last seen supported version was 4
2020-11-03 23:04:28:341 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Constructed client metadata {b21a73a6-2988-47e7-9d38-6c9a313c5315=ClientMetadata{hostInfo=null, consumers=[demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer-2fd342dc-afd4-4c8c-a8b3-95aaa1c5f0c1], state=[activeTasks: ([]) standbyTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevStandbyTasks: ([]) prevAssignedTasks: ([]) capacity: 1]}} from the member subscriptions.
2020-11-03 23:04:28:343 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Starting to validate internal topics {} in partition assignor.
2020-11-03 23:04:28:343 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Completed validating internal topics {} in partition assignor.
2020-11-03 23:04:28:343 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Created repartition topics [] from the parsed topology.
2020-11-03 23:04:28:344 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Starting to validate internal topics {demo-es-1-null-event-seq-state-store-changelog=InternalTopicMetadata(config=UnwindowedChangelogTopicConfig(name=demo-es-1-null-event-seq-state-store-changelog, topicConfigs={}), numPartitions=1)} in partition assignor.
2020-11-03 23:04:28:344 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.InternalTopicManager:? - stream-thread [main] Trying to check if topics [demo-es-1-null-event-seq-state-store-changelog] have been created with expected number of partitions.
2020-11-03 23:04:28:440 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Completed validating internal topics {demo-es-1-null-event-seq-state-store-changelog=InternalTopicMetadata(config=UnwindowedChangelogTopicConfig(name=demo-es-1-null-event-seq-state-store-changelog, topicConfigs={}), numPartitions=1)} in partition assignor.
2020-11-03 23:04:28:440 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Created state changelog topics [InternalTopicMetadata(config=UnwindowedChangelogTopicConfig(name=demo-es-1-null-event-seq-state-store-changelog, topicConfigs={}), numPartitions=1)] from the parsed topology.
2020-11-03 23:04:28:440 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Assigning tasks [0_0] to clients {b21a73a6-2988-47e7-9d38-6c9a313c5315=[activeTasks: ([]) standbyTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevStandbyTasks: ([]) prevAssignedTasks: ([]) capacity: 1]} with number of replicas 0
2020-11-03 23:04:28:441 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.i.StreamsPartitionAssignor:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1-consumer] Assigned tasks to clients as {b21a73a6-2988-47e7-9d38-6c9a313c5315=[activeTasks: ([0_0]) standbyTasks: ([]) assignedTasks: ([0_0]) prevActiveTasks: ([]) prevStandbyTasks: ([]) prevAssignedTasks: ([]) capacity: 1]}.
2020-11-03 23:04:28:635 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] at state PARTITIONS_REVOKED: partitions [input-event-stream-0] assigned at the end of consumer rebalance.
    current suspended active tasks: []
    current suspended standby tasks: []

2020-11-03 23:04:28:635 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from PARTITIONS_REVOKED to PARTITIONS_ASSIGNED
2020-11-03 23:04:28:635 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Creating tasks based on assignment.
2020-11-03 23:04:28:635 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.TaskManager:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Adding assigned tasks as active: {0_0=[input-event-stream-0]}
2020-11-03 23:04:28:641 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Created state store manager for task 0_0 with the acquired state dir lock
2020-11-03 23:04:28:641 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Creating producer client for task 0_0
2020-11-03 23:04:28:938 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Register global stores []
2020-11-03 23:04:29:141 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] partition assignment took 506 ms.
    current active tasks: [0_0]
    current standby tasks: []
    previous active tasks: []

2020-11-03 23:04:29:235 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Initializing stream tasks [0_0]
2020-11-03 23:04:29:239 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Registering state store event-seq-state-store to its state manager
2020-11-03 23:04:29:243 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Transitioning stream task 0_0 to restoring
2020-11-03 23:04:29:639 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StoreChangelogReader:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Start restoring state stores from changelog topics [demo-es-1-null-event-seq-state-store-changelog-0]
2020-11-03 23:04:29:640 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.i.StoreChangelogReader:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] No checkpoint found for task 0_0 state store event-seq-state-store changelog demo-es-1-null-event-seq-state-store-changelog-0 with EOS turned on. Reinitializing the task and restore its state from the beginning.
2020-11-03 23:04:29:739 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Registering state store event-seq-state-store to its state manager
2020-11-03 23:04:29:940 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StoreChangelogReader:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Start restoring state stores from changelog topics [demo-es-1-null-event-seq-state-store-changelog-0]
2020-11-03 23:04:29:940 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.StoreChangelogReader:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Restoring partition demo-es-1-null-event-seq-state-store-changelog-0 from offset 0 to endOffset 44780
2020-11-03 23:04:29:940 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  c.a.t.u.EventSequencerStateRestoreListener:? - event-seq-state-store started restoring for topic : demo-es-1-null-event-seq-state-store-changelog and partition 0 
2020-11-03 23:04:32:341 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  c.a.t.u.EventSequencerStateRestoreListener:? - event-seq-state-store done restoring for topic : demo-es-1-null-event-seq-state-store-changelog and partition 0. Total restored : 44780 
2020-11-03 23:04:32:343 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.AssignedStreamsTasks:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Transitioning stream task 0_0 to running
2020-11-03 23:04:32:434 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  c.a.t.e.EventSequenceProcessor:? - EventSequencer Processor getting initialized with bufferFlushInterval : 100, maxBufferSize : 10000, useExternalKnowledgeTime : true, forwardingLimit: 6000, forwardingIntervalInMillis: 6000
2020-11-03 23:04:32:438 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING
2020-11-03 23:04:38:135 +0000 [kafka-streams-close-thread] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Informed to shut down
2020-11-03 23:04:38:135 +0000 [kafka-streams-close-thread] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from RUNNING to PENDING_SHUTDOWN
2020-11-03 23:04:38:135 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State already transits to PENDING_SHUTDOWN, skipping the run once call after poll request
2020-11-03 23:04:38:135 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Ignoring request to transit from PENDING_SHUTDOWN to PENDING_SHUTDOWN: only DEAD state is a valid next state
2020-11-03 23:04:38:136 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Shutting down
2020-11-03 23:04:38:136 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.TaskManager:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Shutting down all active tasks [0_0], standby tasks [], suspended tasks [], and suspended standby tasks []
2020-11-03 23:04:38:136 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamTask:? - task [0_0] Closing
2020-11-03 23:04:38:138 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.internals.StreamTask:? - task [0_0] Committing
2020-11-03 23:04:38:138 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Flushing all stores registered in the state manager
2020-11-03 23:04:38:138 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.RecordCollectorImpl:? - task [0_0] Flushing producer
2020-11-03 23:04:38:354 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.RecordCollectorImpl:? - task [0_0] Closing producer
2020-11-03 23:04:38:357 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Closing its state manager and all the registered state stores
2020-11-03 23:04:38:357 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] DEBUG o.a.k.s.p.i.ProcessorStateManager:? - task [0_0] Closing storage engine event-seq-state-store
2020-11-03 23:04:38:435 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD
2020-11-03 23:04:38:435 +0000 [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] INFO  o.a.k.s.p.internals.StreamThread:? - stream-thread [demo-es-1-null-b21a73a6-2988-47e7-9d38-6c9a313c5315-StreamThread-1] Shutdown complete