Apache flink 弗林克';s协处理函数不';不定时触发

Apache flink 弗林克';s协处理函数不';不定时触发,apache-flink,flink-streaming,Apache Flink,Flink Streaming,我试着像那样把两条溪流聚合起来 val joinedStream = finishResultStream.keyBy(_.searchId) .connect(startResultStream.keyBy(_.searchId)) .process(new SomeCoProcessFunction) class SomeCoProcessFunction extends CoProcessFunction[SearchFinished, SearchCreated, Search

我试着像那样把两条溪流聚合起来

val joinedStream = finishResultStream.keyBy(_.searchId)
  .connect(startResultStream.keyBy(_.searchId))
  .process(new SomeCoProcessFunction)
class SomeCoProcessFunction extends CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated] {

   override def processElement1(finished: SearchFinished, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#Context, out: Collector[SearchAggregated]): Unit = { 

       // aggregating some "finished" data ...

   }

   override def processElement2(created: SearchCreated, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#Context, out: Collector[SearchAggregated]): Unit = {

       val timerService = ctx.timerService()
       timerService.registerEventTimeTimer(System.currentTimeMillis + 5000)

       // aggregating some "created" data ...
   }

   override def onTimer(timestamp: Long, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#OnTimerContext, out: Collector[SearchAggregated]): Unit = {

       val watermark: Long = ctx.timerService().currentWatermark()
       println(s"watermark!!!! $watermark")

       // clean up the state

   }
然后在
SomeCoProcessFunction
类中处理它们

val joinedStream = finishResultStream.keyBy(_.searchId)
  .connect(startResultStream.keyBy(_.searchId))
  .process(new SomeCoProcessFunction)
class SomeCoProcessFunction extends CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated] {

   override def processElement1(finished: SearchFinished, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#Context, out: Collector[SearchAggregated]): Unit = { 

       // aggregating some "finished" data ...

   }

   override def processElement2(created: SearchCreated, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#Context, out: Collector[SearchAggregated]): Unit = {

       val timerService = ctx.timerService()
       timerService.registerEventTimeTimer(System.currentTimeMillis + 5000)

       // aggregating some "created" data ...
   }

   override def onTimer(timestamp: Long, ctx: CoProcessFunction[SearchFinished, SearchCreated, SearchAggregated]#OnTimerContext, out: Collector[SearchAggregated]): Unit = {

       val watermark: Long = ctx.timerService().currentWatermark()
       println(s"watermark!!!! $watermark")

       // clean up the state

   }
我想要的是在特定时间(5000毫秒)后清除状态,这就是必须使用的
onTimer
。但因为它从未被解雇,我有点问自己我做错了什么

提前谢谢你的提示

更新:

解决方案是这样设置timeService(对费边·休斯克和贝克汉姆来说都是tnx):

timerService.RegisterProcessingTimer(timerService.currentProcessingTime()+5000)


我仍然没有真正弄清楚
timerService.registereventtimeter
做了什么,watermark
ctx.timerService().currentWatermark()
无论EventTimer注册前多长时间,现在总是显示
-9223372036854775808

我看到您使用的是
系统。currentTimeMillis
,它可能与Flink作业使用的
时间特征(事件时间、处理时间、摄取时间)不同


尝试获取事件的时间戳
ctx.timestamp()
,然后在其上添加5000ms。

我看到您使用的是
System.currentTimeMillis
,它可能不同于Flink作业使用的
时间特征(事件时间、处理时间、摄取时间)


尝试获取事件的时间戳
ctx.timestamp()
,然后在其上添加5000ms。

问题是您正在使用处理时间戳(
System.currentTimeMillis+5000
)注册事件时间计时器(
timerService.registerEventTimeTimer

System.currentTimeMillis
返回当前机器时间,但事件时间不是基于机器时间,而是基于从水印计算的时间


您应该注册一个处理计时器,或者使用事件时间戳注册一个事件时间计时器。您可以从作为参数传递给
processElement1()
processElement2()
Context
对象中获取当前水印的时间戳或当前记录的时间戳

问题在于,您正在使用处理时间戳(
System.currentTimeMillis+5000
)注册事件时间计时器(
timerService.registerEventTimeTimer

System.currentTimeMillis
返回当前机器时间,但事件时间不是基于机器时间,而是基于从水印计算的时间

您应该注册一个处理计时器,或者使用事件时间戳注册一个事件时间计时器。您可以从作为参数传递给
processElement1()
processElement2()
Context
对象中获取当前水印的时间戳或当前记录的时间戳