Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/apache-kafka/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache kafka 一分钟的聚合窗口在Kafka 2.4.0中产生意外结果_Apache Kafka_Kafka Consumer Api_Apache Kafka Streams_Confluent Platform - Fatal编程技术网

Apache kafka 一分钟的聚合窗口在Kafka 2.4.0中产生意外结果

Apache kafka 一分钟的聚合窗口在Kafka 2.4.0中产生意外结果,apache-kafka,kafka-consumer-api,apache-kafka-streams,confluent-platform,Apache Kafka,Kafka Consumer Api,Apache Kafka Streams,Confluent Platform,我正在使用来自的代码 使用最新的windowedBy() 然后打印流 stats.foreach((key, value) -> logger.info("Key >>>>> "+ key + " Value => "+value.countTrades)); 我得到以下输出为同一关键点和1分钟窗口。我希望每1分钟打开一次此键的记录。我错过了什么 注意:我也尝试过抑制()。抑制

我正在使用来自的代码 使用最新的windowedBy()

然后打印流

 stats.foreach((key, value) -> logger.info("Key >>>>> "+ key + 
                " Value => "+value.countTrades));
我得到以下输出为同一关键点和1分钟窗口。我希望每1分钟打开一次此键的记录。我错过了什么

注意:我也尝试过抑制()。

抑制()是前进的方向。 本文()解释了如何使用。 使用suppress的唯一警告是,在下一个后续窗口中,必须接收一个新事件,该事件的键与您在groupBy()中使用的键相同


另外,请查看Matthias J.Sax的另一个stackoverflow的评论:

下面是上述git repo中的特定代码,用于将结果作为时间窗口上特定键的单个值

   KStream<String, Trade> source = builder.stream(Constants.STOCK_TOPIC);
    final TimeWindows window = TimeWindows.of(Duration.ofMinutes(1)).grace(Duration.ofSeconds(5));
    KStream<String, TradeStats> stats = 
            source
            .groupByKey()
            .windowedBy(window)
            .<TradeStats>aggregate(() -> new TradeStats(),(k, v, tradestats) -> tradestats.add(v),
                    Materialized.<String, TradeStats, WindowStore<Bytes, byte[]>>as("trade-aggregates")
                            .withValueSerde(new TradeStatsSerde()).withKeySerde(Serdes.String()))
            .suppress(Suppressed.untilWindowCloses(BufferConfig.unbounded()).withName("trade-suppress"))
            .toStream()            
            .map((final Windowed<String> k, final TradeStats v) -> new KeyValue<>(k.key(), v), Named.as("trade-map"));;
    
   stats.foreach((key, value) -> logger.info("Key >>>>> "+ key + " Value => "+value.countTrades));
           
KStream source=builder.stream(Constants.STOCK\u主题);
final TimeWindows window=TimeWindows.of(持续时间分钟(1)).grace(持续时间秒(5));
KStream stats=
来源
.groupByKey()
.windowedBy(窗口)
.aggregate(()->new TradeStats(),(k,v,TradeStats)->TradeStats.add(v),
具体化。as(“贸易总量”)
.withValueSerde(new TradeStatsSerde()).withKeySerde(Serdes.String())
.suppress(supprested.untilwindowcloss(BufferConfig.unbounded()).withName(“交易抑制”))
.toStream()
.map((最终窗口k,最终贸易统计数据v)->新的KeyValue(k.key(),v),命名为.as(“贸易地图”);;
stats.foreach((key,value)->logger.info(“key>>>>>>”+key+“value=>”+value.countTrades));

与您的问题类似的是:
> Line 817: [2021-03-29 15:40:21,444] INFO Key >>>>>> [AES@1617012600000/1617012660000] Value => 19
> Line 823: [2021-03-29 15:40:52,111] INFO Key >>>>>> [AES@1617012600000/1617012660000] Value => 43
> Line 837: [2021-03-29 15:41:24,076] INFO Key >>>>>> [AES@1617012600000/1617012660000] Value => 55
   KStream<String, Trade> source = builder.stream(Constants.STOCK_TOPIC);
    final TimeWindows window = TimeWindows.of(Duration.ofMinutes(1)).grace(Duration.ofSeconds(5));
    KStream<String, TradeStats> stats = 
            source
            .groupByKey()
            .windowedBy(window)
            .<TradeStats>aggregate(() -> new TradeStats(),(k, v, tradestats) -> tradestats.add(v),
                    Materialized.<String, TradeStats, WindowStore<Bytes, byte[]>>as("trade-aggregates")
                            .withValueSerde(new TradeStatsSerde()).withKeySerde(Serdes.String()))
            .suppress(Suppressed.untilWindowCloses(BufferConfig.unbounded()).withName("trade-suppress"))
            .toStream()            
            .map((final Windowed<String> k, final TradeStats v) -> new KeyValue<>(k.key(), v), Named.as("trade-map"));;
    
   stats.foreach((key, value) -> logger.info("Key >>>>> "+ key + " Value => "+value.countTrades));