Java 卡夫卡流:使用时间窗口的聚合结果?
我有以下kafka流应用程序,我需要使用自定义密钥聚合数据。键正在更改,但为了简单起见,我已经开始将键更改为一个字段(SampleMessage中的textId)。在组之后,我需要获得sum(amount)-(amount是SampleMessage类中的一个双字段)。这就是我想到的Java 卡夫卡流:使用时间窗口的聚合结果?,java,apache-kafka,Java,Apache Kafka,我有以下kafka流应用程序,我需要使用自定义密钥聚合数据。键正在更改,但为了简单起见,我已经开始将键更改为一个字段(SampleMessage中的textId)。在组之后,我需要获得sum(amount)-(amount是SampleMessage类中的一个双字段)。这就是我想到的 StreamsBuilder builder = new StreamsBuilder(); builder = builder.addStateStore(Stores.keyValueStoreBuilder
StreamsBuilder builder = new StreamsBuilder();
builder = builder.addStateStore(Stores.keyValueStoreBuilder(
Stores.inMemoryKeyValueStore("myStore"),
Serdes.String(),
Serdes.Long()).withLoggingDisabled());
KTable<String, SampleMessage> sampleMsgKtable = builder.table(TOPIC_NAME,
Consumed.with(Serdes.String(), sampleMsgSerde));
KGroupedTable<String, SampleMessage> groupByAggregation = sampleMsgKtable.groupBy((key, value) -> {
String groupBy = getGroupBy(/**Params **/); // key is now textId
return KeyValue.pair(groupBy, value);
}, Serialized.with(Serdes.String(), sampleMsgSerde));
KTable<String, SampleMessage> reduce = groupByAggregation.reduce(
(current, newValue) -> {
double currentAmount = current.getAmount();
double newAmount = newValue.getAmount();
double total = currentAmount + newAmount;
current.setAmount(total);
return current;
},
(val, agg) -> {
double valAmount = val.getAmount();
double aggAmount = agg.getAmount();
double diff = aggAmount - valAmount;
agg.setAmount(diff);
return agg;
});
KTable<String, String> finalData = myTransformer.transformToString(reduce);
finalData.toStream().to("output");
我被跟踪了
1. textId = x , amount = 45
2. textId = x , amount = 90
3. textId = x , amount = 135
4. textId = x , amount = 180
5. textId = y , amount = 45
现在我想基于时间窗口进行聚合(例如,以5分钟的时间间隔进行聚合)。如何使用KTables实现这一点
1. textId = x , amount = 45
2. textId = x , amount = 90
3. textId = x , amount = 135
4. textId = x , amount = 180
5. textId = y , amount = 45