Mongodb Kafka Connect和Mongo源连接器,具有管道中的查找聚合阶段
我在mongoDB中有以下三个系列:Mongodb Kafka Connect和Mongo源连接器,具有管道中的查找聚合阶段,mongodb,apache-kafka,mongodb-kafka-connector,Mongodb,Apache Kafka,Mongodb Kafka Connector,我在mongoDB中有以下三个系列: //Drinkers { "drinkerId" : "ID1", "name" : "name", "lastName" : "lastname" } //Drinks { "drinkId" : "ID2", "drinkName" :
//Drinkers
{
"drinkerId" : "ID1",
"name" : "name",
"lastName" : "lastname"
}
//Drinks
{
"drinkId" : "ID2",
"drinkName" : "wine",
"alcoholPercent" : 0.12
}
//Happy-Hours
{
"drinkerId" : "ID1",
"drinkId" : "ID1"
}
我还通过下面的mongo管道提交了happy hours集合上的kafka连接器实例
[
{
"$match": {
"operationType": "insert"
}
},
{
"$lookup": {
"from": "drinkers",
"localField": "fullDocument.drinkerId",
"foreignField": "drinkerId",
"as": "drinkerInfo"
}
},
{
"$unwind": "$drinkerInfo"
},
{
"$lookup": {
"from": "drinks",
"localField": "fullDocument.drinkId",
"foreignField": "drinkId",
"as": "drinkInfo"
}
},
{
"$unwind": "$drinkInfo"
}
]
当我查看kafka connect日志时,我看到以下消息:
警告:恢复更改流失败:$lookup不允许在
$changeStream管道20
消息未发布到主题
我真正的使用案例是,我将在drinkers和drinks集合中插入大型mongodb文档,并希望在happy hours集合中插入较小的文档。最终消费者需要饮酒者和饮料信息
我是否有其他方法检索饮酒者和饮料信息,并将详细信息发布到主题中