Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/343.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
如何使用python在kafka consumer中聚合json数据?_Python_Apache Kafka_Kafka Consumer Api_Kafka Producer Api_Kafka Python - Fatal编程技术网

如何使用python在kafka consumer中聚合json数据?

如何使用python在kafka consumer中聚合json数据?,python,apache-kafka,kafka-consumer-api,kafka-producer-api,kafka-python,Python,Apache Kafka,Kafka Consumer Api,Kafka Producer Api,Kafka Python,我在《卡夫卡交易》主题中的数据如下: ConsumerRecord(topic='Transactions',partition=0,offset=3,timestamp=1591277946735,timestamp_type=0,key=None,value={'transaction_id':'9495601361','account_number':14,'transaction_reference':'20070','transaction_datetime':'2020-06-04T

我在《卡夫卡交易》主题中的数据如下:

ConsumerRecord(topic='Transactions',partition=0,offset=3,timestamp=1591277946735,timestamp_type=0,key=None,value={'transaction_id':'9495601361','account_number':14,'transaction_reference':'20070','transaction_datetime':'2020-06-04T19:09:06.735129','amount':260.93},headers=[],校验和=无,序列化的\u键\u大小=-1,序列化的\u值\u大小=160,序列化的\u头\u大小=-1)

ConsumerRecord(topic='Transactions',partition=0,offset=4,timestamp=1591277946736,timestamp_type=0,key=None,value={'transaction_id':'4952940859','account_number':14,'transaction_reference':'44291','transaction_datetime':'2020-06-04T19:09:06.736128','amount'2.82},headers=[],校验和=无,序列化的\u键\u大小=-1,序列化的\u值\u大小=158,序列化的\u头\u大小=-1)

消费者记录(主题为“交易”,分区=0,偏移量=5,时间戳=1591277946737,时间戳类型=0,键=None,值={'transaction_id':'0193362270','account_number':12,'transaction_reference':'96312','transaction_datetime':'2020-06-04T19:09:06.736128','amount':766.95},headers=[],校验和=无,序列化的\u键\u大小=-1,序列化的\u值\u大小=160,序列化的\u头\u大小=-1)

目前编写的消费者代码是:

consumer = KafkaConsumer(bootstrap_servers='localhost:9092',
                                 auto_offset_reset='earliest',
                                 value_deserializer=lambda m: json.loads(m.decode('utf-8')))
consumer.subscribe(['Transactions'])
for message in consumer:
            print (message)

我想要像元组(account_number,sum(amount))这样的输出,我如何实现这一点?

我认为字典可能比元组更有用一些。A将非常适合这个过程

从集合导入defaultdict
accounts=defaultdict(int)
消费者信息:
有效载荷=message.value
账户=有效载荷['account\u number']
金额=有效载荷['amount']
账户[账户]+=金额
打印(帐户)
defaultdict({
"14": 263.75,
"12": 766.95
})
要获取可能要查找的元组,可以在循环后迭代
accounts.items()

用于accounts.items()中的信息:
打印(信息)
("14", 263.75)
("12", 766.95)

C.Nivs,谢谢你的回答,它在defaultdict(,{20:433.13,14:582.7,10:895.99,18:1773.8899999999,2:858.42,8:222.71,13:746.32})中起作用,但是我们不能得到所需的元组输出信息吗?不在你的循环中。但是,您可以在循环之后迭代accounts.items(),这将为您提供所需的元组