Apache flink 操作依赖于什么规则来生成新的事件时间戳

Apache flink 操作依赖于什么规则来生成新的事件时间戳,apache-flink,Apache Flink,我在操作后加入两个流以创建一个新流。代码如下: DataStream<NewTableA> join1 = oldTableADataStream .keyBy(t -> t.getFa3()) .join(tableBDataStream) .where(new oldTableAKeySelector()) .equalTo(new TableBKeySelector()) .windo

我在操作后加入两个流以创建一个新流。代码如下:

DataStream<NewTableA> join1 =
    oldTableADataStream
        .keyBy(t -> t.getFa3())
        .join(tableBDataStream)
        .where(new oldTableAKeySelector())
        .equalTo(new TableBKeySelector())
        .window(EventTimeSessionWindows.withGap(Time.milliseconds(WIN_GAP_TIME)))
        .allowedLateness(Time.milliseconds(allowedLateness))
        .apply(new oldTableAJoinTableBFunc());
        //.assignTimestampsAndWatermarks(new assignTSAndWMLastMax<>(maxOutOfOrderness));


   join1.process(
    new ProcessFunction<NewTableA, NewTableA>() {
      @Override
      public void processElement(NewTableA value, Context ctx, Collector<NewTableA> out)
          throws Exception {
        System.out.println(" NewTableA wmts:" + ctx.timerService().currentWatermark());
        System.out.println(" NewTableA ts:" + ctx.timestamp() + " " + value);
      }
    });
上面的示例中,
oldTableAdatStream
join
tableBDataStream
to
join1
stream on event time

我发现了一个有趣的现象。
join1
中事件的时间戳由flink自动创建

当我创建
OldTableDataStream
tableBDataStream
的测试数据时,我故意设置了所有10000001000000044。但是在join and apply函数之后,新流
join1
中的事件时间戳被flink更改,打印如下:

NewTableA wmts:**1000000042**
 NewTableA ts:**1000000143** NewTableA{PA1=10, A2='a20', FA3=21, B2='b21', B3='b31', FA4=39, C2='null', FC3=null, D2='null', D3='null', ts=**1000000011**}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:**1000000143** NewTableA{PA1=1, A2='a20', FA3=20, B2='b20', B3='b30', FA4=30, C2='null', FC3=null, D2='null', D3='null', ts=**1000000010**}
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=11, A2='a20', FA3=21, B2='b21', B3='b31', FA4=40, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000135 NewTableA{PA1=21, A2='a20', FA3=22, B2='b22', B3='b32', FA4=30, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=38, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000015}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=57, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA ts:1000000143 NewTableA{PA1=2, A2='a20', FA3=20, B2='b20', B3='b30', FA4=31, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=12, A2='a20', FA3=21, B2='b21', B3='b31', FA4=41, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000135 NewTableA{PA1=22, A2='a20', FA3=22, B2='b22', B3='b32', FA4=31, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=58, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=13, A2='a20', FA3=21, B2='b21', B3='b31', FA4=42, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=39, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000015}
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=59, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA ts:1000000135 NewTableA{PA1=23, A2='a20', FA3=22, B2='b22', B3='b32', FA4=32, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000143 NewTableA{PA1=3, A2='a20', FA3=20, B2='b20', B3='b30', FA4=32, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=14, A2='a20', FA3=21, B2='b21', B3='b31', FA4=43, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=40, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000016}
 NewTableA ts:1000000135 NewTableA{PA1=24, A2='a20', FA3=22, B2='b22', B3='b32', FA4=33, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 ......
似乎没有规则创建新事件的时间戳,如何计算10000001431000001381000000135等?似乎与水印无关,因为时间戳水印是100000042,它同时与事件的时间戳不同


操作依据什么规则生成新的事件时间戳,我还没有找到官方指令,谁可以给它们提供链接?

时间窗口创建的输出事件的时间戳设置为窗口内的最大时间戳。因此,它基本上是包含事件的会话的会话边界


由于您已基于FA3为会话窗口设置了关键帧,因此您将获得依赖于FA3的值:20和21的值为1000000143,22的值为1000000135,23和24的值为100000138。

也就是说,新元素的时间戳与生成它的窗口的大小有关,窗口末尾的时间戳是新元素的时间戳?其实正好相反。作为窗口输出创建的元素从窗口获取其时间戳<代码>timestampedCollector.setAbsoluteTimestamp(window.maxTimestamp())
NewTableA wmts:**1000000042**
 NewTableA ts:**1000000143** NewTableA{PA1=10, A2='a20', FA3=21, B2='b21', B3='b31', FA4=39, C2='null', FC3=null, D2='null', D3='null', ts=**1000000011**}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:**1000000143** NewTableA{PA1=1, A2='a20', FA3=20, B2='b20', B3='b30', FA4=30, C2='null', FC3=null, D2='null', D3='null', ts=**1000000010**}
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=11, A2='a20', FA3=21, B2='b21', B3='b31', FA4=40, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000135 NewTableA{PA1=21, A2='a20', FA3=22, B2='b22', B3='b32', FA4=30, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=38, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000015}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=57, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA ts:1000000143 NewTableA{PA1=2, A2='a20', FA3=20, B2='b20', B3='b30', FA4=31, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=12, A2='a20', FA3=21, B2='b21', B3='b31', FA4=41, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000135 NewTableA{PA1=22, A2='a20', FA3=22, B2='b22', B3='b32', FA4=31, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=58, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=13, A2='a20', FA3=21, B2='b21', B3='b31', FA4=42, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=39, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000015}
 NewTableA wmts:1000000042
 NewTableA ts:1000000138 NewTableA{PA1=59, A2='a20', FA3=24, B2='b24', B3='b34', FA4=36, C2='null', FC3=null, D2='null', D3='null', ts=1000000018}
 NewTableA ts:1000000135 NewTableA{PA1=23, A2='a20', FA3=22, B2='b22', B3='b32', FA4=32, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000143 NewTableA{PA1=3, A2='a20', FA3=20, B2='b20', B3='b30', FA4=32, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 NewTableA ts:1000000143 NewTableA{PA1=14, A2='a20', FA3=21, B2='b21', B3='b31', FA4=43, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA ts:1000000138 NewTableA{PA1=40, A2='a20', FA3=23, B2='b23', B3='b33', FA4=34, C2='null', FC3=null, D2='null', D3='null', ts=1000000016}
 NewTableA ts:1000000135 NewTableA{PA1=24, A2='a20', FA3=22, B2='b22', B3='b32', FA4=33, C2='null', FC3=null, D2='null', D3='null', ts=1000000011}
 NewTableA wmts:1000000042
 NewTableA wmts:1000000042
 ......