Java 卡夫卡流分组和连接
我有一个接收记录的卡夫卡流,我想根据特定字段连接消息 流中的消息如下所示:Java 卡夫卡流分组和连接,java,apache-kafka,apache-kafka-streams,Java,Apache Kafka,Apache Kafka Streams,我有一个接收记录的卡夫卡流,我想根据特定字段连接消息 流中的消息如下所示: Key: 2099 Payload{ email: tom@emample.com eventCode: 2099 } 预期产出: key: 2099 Payload{ emails: tom@example, bill@acme.com, jane@example.com } 我可以让水流正常运行,我只是不确定lamda应该包含什么 这就是我到目前为止所做的。我不确定是否应该使用map、aggreg
Key: 2099
Payload{
email: tom@emample.com
eventCode: 2099
}
预期产出:
key: 2099
Payload{
emails: tom@example, bill@acme.com, jane@example.com
}
我可以让水流正常运行,我只是不确定lamda应该包含什么
这就是我到目前为止所做的。我不确定是否应该使用map、aggregate或reduce或这些操作的组合
final StreamsBuilder builder = new StreamsBuilder();
KStream<String, Payload> inputStream = builder.stream(INPUT_TOPIC);
inputStream
.groupByKey()
.windowedBy(TimeWindows.of(TimeUnit.MINUTES.toMillis(300000)))
// Not sure what to do here …..
}).to (OUTPUT_TOPIC );
可能是这样的
inputStream.groupByKey().windowedBy(TimeWindows.of(TimeUnit.MINUTES.toMillis(300000)))
.aggregate(PayloadAggr::new, new Aggregator<String, Payload, PayloadAggr>() {
@Override
public PayloadAggr apply(String key, Payload newValue, PayloadAggr result) {
result.setKey(key);
if(result.getEmails()==null){
result.setEmails(newValue.getEmail());
}else{
result.setEmails(result.getEmails() + "," + newValue.getEmail());
}
return result;
}
}, .../* You serdes and store */}).toStream().to(OUTPUT_TOPIC);
可能是这样的
inputStream.groupByKey().windowedBy(TimeWindows.of(TimeUnit.MINUTES.toMillis(300000)))
.aggregate(PayloadAggr::new, new Aggregator<String, Payload, PayloadAggr>() {
@Override
public PayloadAggr apply(String key, Payload newValue, PayloadAggr result) {
result.setKey(key);
if(result.getEmails()==null){
result.setEmails(newValue.getEmail());
}else{
result.setEmails(result.getEmails() + "," + newValue.getEmail());
}
return result;
}
}, .../* You serdes and store */}).toStream().to(OUTPUT_TOPIC);