Java Flink kafka源在短暂故障后停止提交偏移量

Java Flink kafka源在短暂故障后停止提交偏移量,java,apache-kafka,streaming,apache-flink,flink-streaming,Java,Apache Kafka,Streaming,Apache Flink,Flink Streaming,我正在使用flink1.7和flink连接器卡夫卡版本0.11 Flink检查点被显式关闭,我依靠kafka每5秒自动提交一次偏移量。我不时会看到这个错误: 2019-07-16 08:32:04.273 [JobName] ERROR o.a.kafka.clients.consumer.internals.ConsumerCoordinator - [ConsumerName] Offset commit failed on partition topic-name-0 at offset

我正在使用
flink
1.7
flink连接器卡夫卡
版本
0.11

Flink检查点被显式关闭,我依靠kafka每5秒自动提交一次偏移量。我不时会看到这个错误:

2019-07-16 08:32:04.273 [JobName] ERROR o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Offset commit failed on partition topic-name-0 at offset 7591394545: The request timed out.
2019-07-16 08:32:04.273 [JobName] ERROR o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Offset commit failed on partition topic-name-0 at offset 7591394545: The request timed out.
2019-07-16 08:32:04.310 [JobName] INFO  o.a.kafka.clients.consumer.internals.AbstractCoordinator  - [ConsumerName] Marking the coordinator servername:21000 (id: 2147482313 rack: null) dead
2019-07-16 08:32:04.310 [JobName] INFO  o.a.kafka.clients.consumer.internals.AbstractCoordinator  - [ConsumerName] Marking the coordinator servername:21000 (id: 2147482313 rack: null) dead
2019-07-16 08:32:04.322 [JobName] WARN  o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Asynchronous auto-commit of offsets {topic-name-0=OffsetAndMetadata{offset=7591394751, metadata=''}} failed: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
2019-07-16 08:32:04.322 [JobName] WARN  o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Asynchronous auto-commit of offsets {topic-name-0=OffsetAndMetadata{offset=7591394751, metadata=''}} failed: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
2019-07-16 08:32:04.337 [JobName] WARN  o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Asynchronous auto-commit of offsets {topic-name-0=OffsetAndMetadata{offset=7591394545, metadata=''}} failed: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
2019-07-16 08:32:04.337 [JobName] WARN  o.a.kafka.clients.consumer.internals.ConsumerCoordinator  - [ConsumerName] Asynchronous auto-commit of offsets {topic-name-0=OffsetAndMetadata{offset=7591394545, metadata=''}} failed: Offset commit failed with a retriable exception. You should retry committing the latest consumed offsets.
当这种情况发生时,我检查该组/主题/分区的卡夫卡偏移量,我可以看到它们不再自动提交,如果我必须重新启动作业,它将重播事件发生后的所有数据


我知道flink有没有办法重试提交偏移量?我会增加
request.timeout.ms
参数,但它已经达到305'000 ms

您找到解决此问题的方法了吗?我们使用Flink 1.9,并且遇到了相同的错误。谢谢。自从我们停止使用
flink-connector-kafka-0.11_2.11
(而是开始使用连接到flink发行版的连接器,
flink-connector-kafka_2.11
1.9版)。可以确认切换到
flink-connector-kafka_2.11
后问题消失。