Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka python kafka无法提交消息_Apache Kafka_Kafka Python - Fatal编程技术网

Apache kafka python kafka无法提交消息

Apache kafka python kafka无法提交消息,apache-kafka,kafka-python,Apache Kafka,Kafka Python,我创建了一个新主题,并编写了一个函数来使用消息: consumer = KafkaConsumer(topic, bootstrap_servers=broker_list, auto_offset_reset='earliest', group_id='test', enable_auto_commit=True) for msg in consumer: print(msg.value) time.sleep(1) 我使用以下函数检查每个分区的提交状态: def get_c

我创建了一个新主题,并编写了一个函数来使用消息:

consumer = KafkaConsumer(topic, bootstrap_servers=broker_list, auto_offset_reset='earliest', group_id='test', enable_auto_commit=True)
for msg in consumer:
    print(msg.value)
    time.sleep(1)
我使用以下函数检查每个分区的提交状态:

def get_current_offset(topic, group):

    consumer = KafkaConsumer(
            bootstrap_servers=broker_list,
            group_id=group,
            enable_auto_commit=False
        )   

    for p in consumer.partitions_for_topic(topic):
        tp = TopicPartition(topic, p)
        consumer.assign([tp])
        committed = consumer.committed(tp)
        consumer.seek_to_end(tp)
        last_offset = consumer.position(tp)

        if not committed:
            committed = 0 
        print("topic: %s partition: %s committed: %s last: %s lag: %s" % (topic, p, committed, last_offset, (last_offset - committed)))

    consumer.close(autocommit=False)
问题是每当我使用消息时,
get\u current\u offset
的输出是:

topic: zhihu_comment partition: 0 committed: 0 last: 29199 lag: 29199
topic: zhihu_comment partition: 1 committed: 0 last: 29089 lag: 29089
topic: zhihu_comment partition: 2 committed: 0 last: 29149 lag: 29149
这表明没有提交任何消息。我觉得奇怪的是,另一个话题,以同样的方式消费,可以正常提交

topic: zhihu_profile partition: 0 committed: 699879 last: 699879 lag: 0
topic: zhihu_profile partition: 1 committed: 697061 last: 697061 lag: 0
topic: zhihu_profile partition: 2 committed: 697127 last: 697127 lag: 0
这个话题有什么问题吗

更新: 我试着从一开始就使用
kafka控制台消费者--引导服务器设备1:9092--主题智虎评论--组测试
要消费,消息将正常提交

所以,我怀疑这是因为卡夫卡python的问题