Apache kafka python kafka无法提交消息
我创建了一个新主题,并编写了一个函数来使用消息:Apache kafka python kafka无法提交消息,apache-kafka,kafka-python,Apache Kafka,Kafka Python,我创建了一个新主题,并编写了一个函数来使用消息: consumer = KafkaConsumer(topic, bootstrap_servers=broker_list, auto_offset_reset='earliest', group_id='test', enable_auto_commit=True) for msg in consumer: print(msg.value) time.sleep(1) 我使用以下函数检查每个分区的提交状态: def get_c
consumer = KafkaConsumer(topic, bootstrap_servers=broker_list, auto_offset_reset='earliest', group_id='test', enable_auto_commit=True)
for msg in consumer:
print(msg.value)
time.sleep(1)
我使用以下函数检查每个分区的提交状态:
def get_current_offset(topic, group):
consumer = KafkaConsumer(
bootstrap_servers=broker_list,
group_id=group,
enable_auto_commit=False
)
for p in consumer.partitions_for_topic(topic):
tp = TopicPartition(topic, p)
consumer.assign([tp])
committed = consumer.committed(tp)
consumer.seek_to_end(tp)
last_offset = consumer.position(tp)
if not committed:
committed = 0
print("topic: %s partition: %s committed: %s last: %s lag: %s" % (topic, p, committed, last_offset, (last_offset - committed)))
consumer.close(autocommit=False)
问题是每当我使用消息时,get\u current\u offset
的输出是:
topic: zhihu_comment partition: 0 committed: 0 last: 29199 lag: 29199
topic: zhihu_comment partition: 1 committed: 0 last: 29089 lag: 29089
topic: zhihu_comment partition: 2 committed: 0 last: 29149 lag: 29149
这表明没有提交任何消息。我觉得奇怪的是,另一个话题,以同样的方式消费,可以正常提交
topic: zhihu_profile partition: 0 committed: 699879 last: 699879 lag: 0
topic: zhihu_profile partition: 1 committed: 697061 last: 697061 lag: 0
topic: zhihu_profile partition: 2 committed: 697127 last: 697127 lag: 0
这个话题有什么问题吗
更新:
我试着从一开始就使用kafka控制台消费者--引导服务器设备1:9092--主题智虎评论--组测试
要消费,消息将正常提交
所以,我怀疑这是因为卡夫卡python的问题